Changes between Version 11 and Version 12 of Stack_Sky_Levels

Timestamp:

May 12, 2014, 5:37:16 PM (12 years ago)

Author:

watersc1

Comment:

--

Stack_Sky_Levels

@@ +1 @@
+== 2014-05-07 ==
+
+After a suggestion from Gene and a series of tests, I've come to the
+conclusion that the small residual chip background is a function of
+the image quantization/compression code.  The following table shows
+the background mean/sigma values for two warps for a series of tests.
+
+The first two rows are the entries from the gpc1 database for the
+skycell and primary chip image (this skycell are comprised primarily
+of one OTA) in the test.
+
+All subsequent tests were performed with identical random seed (1234).
+The seed.passX rows are the background values generated in ppImage
+with a three-pass change (fit background, subtract, repeat).  They
+seem to confirm that the statistic is working just fine.
+
+Next is the in-and-out test, where after writing the image, psphot is
+used to remeasure the background.  This provides the ~0.3 count offset
+seen in previous tests.  To confirm that this is a compression
+artifact, I repeated with an image saved by ppImage uncompressed.
+There are three sigma based scaling methods defined in psLib, and I
+checked that they exhibit biases around zero that make some sort of
+sense (the positive scale is what is used in the normal compressed
+image, so that is shown in the in-and-out test).
+
+Finally, I noticed that the statistic used to generate the mean/sigma
+values for the scaling is psLib was the "robust" median/stddev, not
+the "fitted" that is used for the background tests.  Changing this and
+repeating the tests did not improve the result significantly.
+
+
+The way the scaling flips around zero (and that the "both" is still
+offset in one direction) makes me suspect that there's an off-by-one
+error in the scaling code.  I haven't yet tracked it down, but I
+wanted to get this status note out.
+
+||#test           ||warp.964581  ||||                   warp.964593||||
+||chip.bg        || 25.1021       ||  17.3215    ||     28.8274   ||      8.38539
+||warp.bg        || 0.366421      ||  6.898341   ||     0.357391  ||      6.962276
+||seed.pass1.bg  || 25.729        ||  8.0774     ||     28.7602    ||     8.075996
+||seed.pass2.bg  || 0.0147        ||  8.0745     ||     0.000294  ||      8.07090
+||seed.pass3.bg  || 0.000620      ||  8.0765     ||     0.004085  ||      8.066377
+||in-and-out     || 0.281408       || 7.734210   ||     0.296758  ||      25.729009
+||io.uncomp      || 0.013544      ||  8.066099   ||     -0.002336 ||      8.080612
+||io.both        || 0.182691      ||  7.497421   ||     0.193386  ||      7.664834
+||io.neg         || -0.307696     ||  7.727381   ||     -0.230592 ||      7.777576
+||io.fitted      || 0.298210      ||  7.775235   ||     0.253315  ||      7.689663
+||io.fitted-both || 0.159112      ||  7.453220   ||     0.142325  ||      7.238388
+||io.fitted.int  || 0.027772      ||  8.151440   ||     -0.108624 ||      7.896051
+
+
+== 2014-05-01 ==
+
+After looking through the code on this, I think I've finally sorted
+out the source of this background.  In order to ensure that all stacks
+have a zeropoint of 25, the input fluxes are normalized, such that:
+
+{{{
+f_stack = \sum (f_i * w_i)/sum(w_i) * n_i
+with
+n_i = 10**( -0.4 * ([z_filter + eps_seeing + 2.5 log10(t_i)] -
+                       [z_stack + 2.5 log10(t_stack)]))
+  ~ N * 10**(-0.4 * [z_filter - z_stack]);
+}}}
+
+This means that any residual background is the stack is multiplied by
+N and a zeropoint factor that ranges from 1.3 (r) to 5.1 (y).  This
+means for a typical g-band 8-input stack, the residual background is
+multipled by ~13.5.
+
+A typical g warp seems to have a residual background of ~.3-.4 with a
+sigma of ~7-8.  The current psStats implementation simply cannot find
+that small of a signal.  As we start stacking, the normalization
+scaling pushes up that signal much faster than the image variance:
+
+|| Ninput||      gauss_mean||      gauss_sigma||
+|| 4 ||  2.22026||         35.853||
+|| 8 ||  5.57725||         51.721||
+|| 12 ||         8.83167||         50.0609||
+|| 19||          14.4736||         82.931||
+
+We do not generate a background model of the final stack, but this is
+done prior to doing the photometry.  As the s/n of the background is
+much higher in the stacks, this can identify and subtract it off,
+yielding measured SKY values that are non-zero.
+
+
+
+Stacks assign weights the input exposures, and this weight has a term related to the difference between the input zeropoint and the target zeropoint of 25.0.
+
+#test           warp.964581                     warp.964593
+chip.bg         25.1021         17.3215         28.8274         8.38539
+warp.bg         0.366421        6.898341        0.357391        6.962276
+seed.pass1.bg   25.729          8.0774          28.7602         8.075996
+seed.pass2.bg   0.0147          8.0745          0.000294        8.07090
+seed.pass3.bg   0.000620        8.0765          0.004085        8.066377
+in-and-out      0.281408        7.734210        0.296758        25.729009
+io.uncomp       0.013544        8.066099        -0.002336       8.080612
+io.both         0.182691        7.497421        0.193386        7.664834
+io.neg          -0.307696       7.727381        -0.230592       7.777576
+io.fitted       0.298210        7.775235        0.253315        7.689663
+io.fitted-both  0.159112        7.453220        0.142325        7.238388
+io.fitted.int   0.027772        8.151440        -0.108624       7.896051
+
+
 == 2013-12-04 ==