Changeset 41803 for branches/eam_branches/ipp-dev-20210817

Timestamp:

Sep 2, 2021, 11:31:43 AM (5 years ago)

Author:

eugene

Message:

adding some in-line notes on pattern row correction

File:

• branches/eam_branches/ipp-dev-20210817/psModules/src/detrend/pmPattern.c (modified) (1 diff)

branches/eam_branches/ipp-dev-20210817/psModules/src/detrend/pmPattern.c

@@ -9 +9 @@
 #define PATTERN_ROW_BKG_FIX 1
 
+/* some in-line notes:
+
+   patternMaskRow sets the data value to NAN, inconsistent with new plan
+   
+   here is the outline of pmPatternRow
+
+   * at this point, we have already done overscan subtraction, right?
+
+   * measure stats on the full cell (MEDIAN, STDEV)
+   ** subsample?
+   ** if it fails, it masks the entire cell and set the value to NAN
+
+   * calculate an upper and lower threshold (median +/- T * sigma)
+   * define a normalized x-coordinate ('index') : 
+   ** see note below on chebys
+
+   * each row is treated independently
+   * pixels are masked for the fit if they are out-of-range 
+     or if they are already masked
+
+   ** note that the clipping threshold will be larger if there 
+      are pixels which have astronomical structures
+      
+      a possible better option would be to set the threshold based on the median
+      and a sigma calculated from Poisson stats (do we know the gain?)
+      
+   ** fit is allowed to proceed if even N+1 pixels exist, which is clearly too low
+   
+   ** Remaining pixels are fitted with clip-fit 
+
+   ** solution is subtracted from the data
+   (this is implemented with psPolynomial1DEvalVector)
+   perhaps faster if we fixed the order to 2 and hardwired the result
+   
+   * after each row is fitted, the intercept (A value) is fitted
+   as a function of the y-coordinate and the result is subtracted
+
+   * the slope value is also fitted as a function of the column 
+     and added back in -- I'm not sure I understand this step.
+
+   *****************
+
+   ** what we calculate are related to chebychevs (domain is -1 : +1)
+   *** T0(x) = 1
+   *** T1(x) = x
+   *** T2(x) = 2x^2 - 1
+
+   *** we calculate y = A + Bx + Cx^2
+
+   a_0 + a_1 x + a_2 (2x^2 - 1) = A + B + Cx^2
+
+   a_1       = B
+   a_0 - a_2 = A
+   2 a_2     = C
+
+   a_0       = A + C/2
+   a_1       = B
+   a_2       = C/2
+
+   *****************
+   
+   I have 3 goals in re-working the code:
+   
+   1) improve overall speed
+   2) improve reliability of the fit
+   3) skip fit if we can
+
+   Let's assume the signal in the cell is light + bias drift
+
+   The bias drift has an amplitude of ~5 - 10 DN
+
+   That makes a detectable source with ~N * a few counts (multiple pixels in a row)
+   
+   So, the effective flux is ~10 * 5 = 50 DN
+   for which sky level is this value - 3 sigma?
+   
+   50 / sqrt(sky sigma^2 * effective area)
+
+   area ~ 5pixels, sky sigma^2 = sky
+
+   10 * Npix / sqrt(sky * Npix) = 3
+
+   Npix = sky * (S/N)^2 / (peak^2)
+   
+   sky = Npix * peak^2 / SN^2
+
+   if (sky < Npix * peak^2 / SN^2), we should skip:
+   
+   Npix ~ 5
+   peak ~ 10
+   SN ~ 3 (or even less)
+
+   sky < 5 * 100 / 9 = 55 or so
+
+ */
 
 // Mask a row as bad