Summary of MD09.PV1_deepstack_tests

(MediumDeepFields summary page)

Recent deep stacks test include (label/data_group)

• MD09.deeptest.20120727: best attempt to make a 3x input stack over the refstacks with constrained PSF on the convolved stacks for staticsky stack photometry and to avoid critical failures in order to produce a full field stack in each filter. As part of the initial MD.GR0 reprocessing, details can be found at MD.GR0, and only uses available exposures from 2009-2011 reprocessing (also PV1 class).
• stack-o-stacks: an idea to try stacking night stacks in order to reduce the large processing time required in deepstacks. Local tests were run -- while the code seems to make them and in significantly less time, it is unclear how good the result is and does break some of the extra output products such as the .num.fits and .exp.fits without a good amount of development work.
• MD09.deeptest.20130330: mostly failed attempt at increasing the input number from 20120727 due to bugs in ppStack
• MD09.alldeeptest.20130507: a sub-field set of skycells with everything going in, only failed PSF matching rejected. (using improved ppStack version, see stacking_coverage.20130307)
• MD09.cutdeeptest.20130508: a sub-field set of skycells using exposures from 2010-2012 with modest FWHM, background, ZPT cuts (see section below)
• MD09.deeptest.20130509: full field set of all 2009-2012 exposures with modest FWHM, background, ZPT cuts.

Exposures from 2012 by default are PV1, 2009-2011 are GR0 reprocessing that are also PV1 class.

MD09.alldeeptest.20130507

Include all MD09 data from 2009-04 through end of last season Dec. 2012.

Test skycells:

• 027 (past problem edge mix of old 2009 and newer dither data, 1 galaxy from Thilker list)
• 045 (2 galaxies)
• 055 (common test skycell, center with PSF issues, 1 galaxy)
• 068 (another near edge, 2 galaxies)

N input and rejected warps:

| stack_id | label                     | skycell_id  | filter  | fault | quality | reject_images | dtime_script | count(warp_id) |
+----------+---------------------------+-------------+---------+-------+---------+---------------+--------------+----------------+
|  2237787 | MD09.alldeeptest.20130507 | skycell.027 | g.00000 |     0 |       0 |            35 |        44117 |            577 | 
|  2237788 | MD09.alldeeptest.20130507 | skycell.027 | i.00000 |     0 |       0 |            36 |        44957 |            583 | 
|  2237789 | MD09.alldeeptest.20130507 | skycell.027 | r.00000 |     0 |       0 |           259 |        40616 |            567 | 
|  2237790 | MD09.alldeeptest.20130507 | skycell.027 | y.00000 |     0 |       0 |           388 |        33102 |            562 | 
|  2237791 | MD09.alldeeptest.20130507 | skycell.027 | z.00000 |     0 |       0 |           317 |        51544 |            654 | 
|  2237792 | MD09.alldeeptest.20130507 | skycell.045 | g.00000 |     0 |       0 |            22 |        82983 |            578 | 
|  2237793 | MD09.alldeeptest.20130507 | skycell.045 | i.00000 |     0 |       0 |            38 |        83354 |            584 | 
|  2237794 | MD09.alldeeptest.20130507 | skycell.045 | r.00000 |     0 |       0 |            24 |        75814 |            567 | 
|  2237795 | MD09.alldeeptest.20130507 | skycell.045 | y.00000 |     0 |       0 |           249 |        66896 |            568 | 
|  2237796 | MD09.alldeeptest.20130507 | skycell.045 | z.00000 |  NULL |    NULL |          NULL |         NULL |            657 | 
|  2237797 | MD09.alldeeptest.20130507 | skycell.055 | g.00000 |     0 |       0 |            14 |        78568 |            578 | 
|  2237798 | MD09.alldeeptest.20130507 | skycell.055 | i.00000 |     0 |       0 |            33 |        83524 |            584 | 
|  2237799 | MD09.alldeeptest.20130507 | skycell.055 | r.00000 |     0 |       0 |            17 |        76823 |            567 | 
|  2237800 | MD09.alldeeptest.20130507 | skycell.055 | y.00000 |     0 |       0 |            96 |        82326 |            566 | 
|  2237801 | MD09.alldeeptest.20130507 | skycell.055 | z.00000 |  NULL |    NULL |          NULL |         NULL |            657 | 
|  2237802 | MD09.alldeeptest.20130507 | skycell.068 | g.00000 |     0 |       0 |            67 |        18716 |            338 | 
|  2237803 | MD09.alldeeptest.20130507 | skycell.068 | i.00000 |     0 |       0 |            54 |        20887 |            328 | 
|  2237804 | MD09.alldeeptest.20130507 | skycell.068 | r.00000 |     0 |       0 |            84 |        20893 |            327 | 
|  2237805 | MD09.alldeeptest.20130507 | skycell.068 | y.00000 |     0 |       0 |           288 |         6864 |            301 | 
|  2237806 | MD09.alldeeptest.20130507 | skycell.068 | z.00000 |     0 |       0 |           115 |        19176 |            353 | 

• label=data_group on datastore and postage stamp server. If dtime_script NULL, then the stack is still running.
• stack pixels to the datastore under ​ps1-deeptest

MD09.cutdeeptest.20130508

Include all MD09 data from April 2009 through end of last season Dec. 2012, but impose exposure level cuts as suggested by Ken Chambers:

• FWHM<=10 pixels (2.5") is ~9.77 pixel in exposure stats (table numbers using iq_fwhm_major)
• elong<=1.5
• dZPT<=0.33 mag, so using mean ZPT grizy >= 24.09, 24.32, 24.23, 23.85, 22.89
• background grizy <= 600, 800, 2500, 4000, 6000
• drop 2009 data as may have a variety of issues
• using 2010+cuts in table below for cutdeeptest.20130508. 2010+ is all data 2010-2012, cuts is all data 2009-2012 with the input cuts applied (numbers for edge skycells can be greatly less).

filter	all	2010+	cuts	2010+cuts	comment
g	584	545	500	464	lose ~120
r	571	530	498	457	lose ~110
i	599	525	526	462	lose ~140, driven by what appears evenly by all cuts..
z	660	599	598	539	lose ~120 ok
y	575	552	421	401	lose ~170, seem a problem mainly with many of poorer ZPT from marginal weather observations

Test skycells (same set):

• 027 (past problem edge mix of old 2009 and newer dither data, 1 galaxy)
• 045 (2 galaxies)
• 055 (common test skycell, center with PSF issues, 1 galaxy)
• 068 (another near edge, 2 galaxies)

| stack_id | label                     | skycell_id  | filter  | fault | quality | reject_images | dtime_script | count(warp_id) |
+----------+---------------------------+-------------+---------+-------+---------+---------------+--------------+----------------+
|  2237807 | MD09.cutdeeptest.20130508 | skycell.027 | g.00000 |     0 |       0 |            71 |        26885 |            472 | 
|  2237808 | MD09.cutdeeptest.20130508 | skycell.027 | r.00000 |     0 |       0 |           132 |        35485 |            462 | 
|  2237809 | MD09.cutdeeptest.20130508 | skycell.027 | i.00000 |     0 |       0 |            43 |        31509 |            470 | 
|  2237810 | MD09.cutdeeptest.20130508 | skycell.027 | z.00000 |     0 |       0 |            56 |        34626 |            531 | 
|  2237811 | MD09.cutdeeptest.20130508 | skycell.027 | y.00000 |     0 |       0 |           124 |        22651 |            405 | 
|  2237812 | MD09.cutdeeptest.20130508 | skycell.045 | g.00000 |     0 |       0 |            45 |        55381 |            473 | 
|  2237813 | MD09.cutdeeptest.20130508 | skycell.045 | r.00000 |     0 |       0 |            35 |        63818 |            462 | 
|  2237814 | MD09.cutdeeptest.20130508 | skycell.045 | i.00000 |     0 |       0 |            30 |        61355 |            471 | 
|  2237815 | MD09.cutdeeptest.20130508 | skycell.045 | z.00000 |     0 |       0 |            39 |        73388 |            532 | 
|  2237816 | MD09.cutdeeptest.20130508 | skycell.045 | y.00000 |     0 |       0 |            63 |       169391 |            405 | 
|  2237817 | MD09.cutdeeptest.20130508 | skycell.055 | g.00000 |     0 |       0 |            37 |        54054 |            473 | 
|  2237818 | MD09.cutdeeptest.20130508 | skycell.055 | r.00000 |     0 |       0 |            44 |        65216 |            462 | 
|  2237819 | MD09.cutdeeptest.20130508 | skycell.055 | i.00000 |     0 |       0 |            22 |        63907 |            471 | 
|  2237820 | MD09.cutdeeptest.20130508 | skycell.055 | z.00000 |     0 |       0 |            38 |        72188 |            532 | 
|  2237821 | MD09.cutdeeptest.20130508 | skycell.055 | y.00000 |     0 |       0 |            44 |        50172 |            405 | 
|  2237822 | MD09.cutdeeptest.20130508 | skycell.068 | g.00000 |     0 |       0 |            50 |        11495 |            262 | 
|  2237823 | MD09.cutdeeptest.20130508 | skycell.068 | r.00000 |     0 |       0 |            50 |        10491 |            247 | 
|  2237824 | MD09.cutdeeptest.20130508 | skycell.068 | i.00000 |     0 |       0 |            47 |         9472 |            234 | 
|  2237825 | MD09.cutdeeptest.20130508 | skycell.068 | z.00000 |     0 |       0 |            74 |         9465 |            261 | 
|  2237826 | MD09.cutdeeptest.20130508 | skycell.068 | y.00000 |     0 |       0 |           121 |         5533 |            205 | 

• label=data_group on datastore and postage stamp server
• stack pixels to the datastore under ​ps1-deeptest

MD09.deeptest.20130509

Setup with same cuts as above in MD09.cutdeeptest.20130508, but includes 2009-2012 (cuts in table).

• running -- status as of 5/12

  | label                  | filter  | avg(reject_images) | avg(dtime_script) | count(stack_id) |
  +------------------------+---------+--------------------+-------------------+-----------------+
  | MD09.deeptest.20130509 | g.00000 |            54.9254 |   38529.208954996 |              67 | /73
  | MD09.deeptest.20130509 | r.00000 |            32.1538 |   12320.846153846 |              13 | /73