Summary for MD.pv2.iqstack.20140824

This is a summary of the MD PV2 IQ stack using the PV2 update reprocessing (as started with MD04 in MD04.pv2_stacktest.20130903) for all MD exposures through the end of the survey, April 2014. Because MD.PV2 is an update reprocessing method, there is a mix of photometry from PV1 and PV2 (as tracked by the data_group name summarized in the MD.PV2 wiki page), while the warp pixels used for the stacking is of PV2 quality (using the same processing tag ipp-20130712 as with LAP/3PI).

The processing sequence will be MD04, MD09, then continue in order MD01-10.

  • pixel rejection bug fixes and config modifications MDdeeptest_pixrej_20140403
  • staticsky and possibly skycal changes (i.e., MD04 has had a gross offset at least in i-band)
  • all processing done using the ipp-pv3-20140717 tag now with specific config MD_IQ_STACK_1DG and new refcat (20140713)

Label/data_group names will be uniform now -- PV2 IQ stacks -- and will be summarized here and not in the subsections

  • label/data_group:
    • MD<field>.pv2.iqstack.20140824 --
  • staticsky label/data_group: (same format as other stacks -- Nx<rundate>/<#>x<rundate>)
    • MD<field>.pv2.iqstack.20140824.Nx20140824/MD04.pv2.iqstack.20140824.<#>x20140824 --
  • skycal label/data_group:
    • MD<field>.pv2.iqstack.20140824.Nx20140824.cal20140824

As for all stacks, basic cuts will be used to exclude poorer data (contribute less in weighted stacks, potential for more junk, waste resources) -- likely will add summary plots on analysis page for these params for revision -- basically similar to those used for the refstack.20140

  • pre-June 2010 exposures fully excluded
    • significant camera change prior 1/2010
    • baffle modifications prior 6/2010
  • ZPT 24.3, 24.4, 24.3, 24.0, 23.0 (grizy each by mean exposure value, same as refstacks)
  • background < 300, 600, 2000, 3000, 5000 (grizy each by mean exposure value)
  • elongation < 1.3 (grizy by exposure value)
  • FWHM < 10 pixels (grizy each by skycell, not by full exposure value) -- unlike <7 pixels for refstack, need <10 so edge skycells are not completely obliterated.
  • no additional clipping of FWHM of larger inputs (like for refstack) since on early rising edge of FWHM distribution that can cause irregular rejection of inputs and leave large holes in coverage

Data can be found on the datastore and postage stamp server (PSS) --

  • datastore: full image/pixel bundles, staticsky and skycal stack photometry
    • ps1-md-iqstack -- these current full PV2 image/pixel bundles
    • ps1-md-staticsky -- staticsky and skycal stack photometry catalog bundles
  • PSS: selected desired image/pixels
    • gpc1 DB and original PSS interfaces use the label/data_group above
    • PSI/PSS access -- use the release label/data_group for the desired stack in the pull-down menu of format -- MD<field>.pv2.iqstack.20140824.cal20140824

Any problems and/or issues need to be identified quickly so that changes can be made before too many MD stacks are made.

  • skycell loss due to quality issue will be unresolved for PV2 stacks
  • background bias/offset -- internal correction to the warps applied in the ipp-pv3-20140717 tag but may still not fully corrected (minimal for smaller N input stacks)

Similar to GR0 there will be a summary for each field, but unlike GR0, will try and keep analysis and issues to other pages to reduce the length and load time for this summary page.

Status:

  • 7/x -- testing for FWHM and depth
  • 8/20 -- MD04.iqstack.20140820 test run, ~30-35 inputs (manually set FWHM limit, relaxed ZPT,BG,elong cuts)
  • 8/21 -- MD04.pv2.iqstack.20140821 test run, reduced to 25 inputs but excess, irregular FWHM clipping from config so could be 15-25 inputs
  • 8/24 -- MD04.pv2.iqstack.20140824 started using tag ipp-pv3-20140717 with MD_IQ_STACK_1DG config
  • 8/25 -- MD09.pv2.iqstack.20140824 run and verfified
  • 8/26--8/27 -- MD01,02,03,05,06,07,08,10 run when nightly processing not demanding the data nodes
  • 8/28 to be finished and distributed, stack photometry finished
  • finished

pre-test runs

MD04.iqstack.20140820 -- ~30 input for better filling of pixels on the field edges

| stack_id | label                 | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+-----------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3416208 | MD04.iqstack.20140820 | g.00000 | skycell.034 |         4000 |             0 |             32 |  0.987443 | 
|  3416268 | MD04.iqstack.20140820 | r.00000 | skycell.034 |         3396 |             0 |             31 |  0.987334 | 
|  3416328 | MD04.iqstack.20140820 | i.00000 | skycell.034 |         3846 |             0 |             33 |  0.987197 | 
|  3416388 | MD04.iqstack.20140820 | z.00000 | skycell.034 |         3644 |             0 |             33 |  0.987285 | 
|  3416448 | MD04.iqstack.20140820 | y.00000 | skycell.034 |         3580 |             0 |             30 |  0.987414 | 

MD04.pv2.iqstack.20140821 -- 25 input but problem with extra rejection of inputs based on larger input FWHM on field edges

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3416510 | MD04.pv2.iqstack.20140821 | g.00000 | skycell.034 |         3292 |             0 |             25 |  0.987419 | 
|  3416570 | MD04.pv2.iqstack.20140821 | r.00000 | skycell.034 |         2642 |             0 |             25 |  0.987343 | 
|  3416630 | MD04.pv2.iqstack.20140821 | i.00000 | skycell.034 |         2986 |             0 |             25 |  0.987158 | 
|  3416690 | MD04.pv2.iqstack.20140821 | z.00000 | skycell.034 |         2714 |             0 |             25 |  0.987238 | 
|  3416750 | MD04.pv2.iqstack.20140821 | y.00000 | skycell.034 |         2658 |             0 |             25 |  0.987393 |

| stack_id | label                 | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+-----------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3416191 | MD04.iqstack.20140820 | g.00000 | skycell.012 |          428 |             6 |             17 |  0.155209 | 
|  3416251 | MD04.iqstack.20140820 | r.00000 | skycell.012 |          528 |             5 |             16 |  0.165574 | 
|  3416311 | MD04.iqstack.20140820 | i.00000 | skycell.012 |          966 |             0 |             21 |  0.175924 | 
|  3416371 | MD04.iqstack.20140820 | z.00000 | skycell.012 |         1048 |             0 |             18 |  0.177031 | 
|  3416431 | MD04.iqstack.20140820 | y.00000 | skycell.012 |          781 |             1 |             18 |   0.18823 | 

Both on datastore under ps1-deeptest with the label/data_group noted above.


MD04

PV2 camera & warps -- PV2 warps are not cleaned up and will also be a accessible via the PSS/PSPS interface more quickly (release name MD04.PV2).

  • label: MD04.pv2.20130903

PV2 IQ stack:

  • label/data_group:
    • MD04.iqstack.20140820 -- test version
    • MD04.pv2.iqstack.20140821 -- test version
    • MD04.pv2.iqstack.20140824 -- final version, also label for postage stamp server
  • staticsky label/data_group: (same format as other stacks -- Nx<rundate>/<#>x<rundate>)
    • MD04.pv2.deepstack.20140506.Nx20140511/MD04.pv2.iqstack.20140824.<#>x20140824 --
  • skycal label/data_group:
    • MD04.pv2.iqstack.20140824.Nx20140824.cal20140824 -- PV3 tag with new fixed refcat (20140713) and improved zeropoint recalibration config

Summary of stack produced:

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3416811 | MD04.pv2.iqstack.20140824 | g.00000 | skycell.035 |         2350 |             0 |             25 |  0.987373 | 
|  3416871 | MD04.pv2.iqstack.20140824 | r.00000 | skycell.035 |         2170 |             0 |             25 |  0.987292 | 
|  3416931 | MD04.pv2.iqstack.20140824 | i.00000 | skycell.035 |         2430 |             0 |             25 |  0.987131 | 
|  3416991 | MD04.pv2.iqstack.20140824 | z.00000 | skycell.035 |         2088 |             0 |             25 |  0.987219 | 
|  3417051 | MD04.pv2.iqstack.20140824 | y.00000 | skycell.035 |         2139 |             0 |             25 |  0.987373 | 

Problems:

  • no quality faults
  • poor pixels/regions -- to be listed (by request), generally all edge skycells have varying levels of irregular pixel coverage losses

MD09

PV2 camera & warps -- PV2 warps are not cleaned up and will also be a accessible via the PSS/PSPS interface more quickly (release name MD09.PV2).

  • label: MD09.pv2.20131030

Summary of stack produced:

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3417110 | MD09.pv2.iqstack.20140824 | g.00000 | skycell.035 |         2340 |             0 |             25 |   0.98679 | 
|  3417170 | MD09.pv2.iqstack.20140824 | r.00000 | skycell.035 |         1980 |             0 |             25 |  0.986837 | 
|  3417230 | MD09.pv2.iqstack.20140824 | i.00000 | skycell.035 |         2379 |             0 |             25 |  0.985342 | 
|  3417290 | MD09.pv2.iqstack.20140824 | z.00000 | skycell.035 |         2076 |             0 |             25 |  0.986922 | 
|  3417350 | MD09.pv2.iqstack.20140824 | y.00000 | skycell.035 |         2327 |             0 |             25 |  0.987143 | 

Problems:

  • no quality faults
  • poor pixels/regions need to be listed (by request)

MD01

PV2 camera & warps -- PV2 warps are not cleaned up and will also be a accessible via the PSS/PSPS interface more quickly (release name MD01.PV2).

  • label: MD01.pv2.20131031

Summary of stack produced:

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3417411 | MD01.pv2.iqstack.20140824 | g.00000 | skycell.035 |         2357 |             0 |             25 |  0.987412 | 
|  3417471 | MD01.pv2.iqstack.20140824 | r.00000 | skycell.035 |         2069 |             0 |             25 |   0.98725 | 
|  3417531 | MD01.pv2.iqstack.20140824 | i.00000 | skycell.035 |         2293 |             0 |             25 |  0.987122 | 
|  3417591 | MD01.pv2.iqstack.20140824 | z.00000 | skycell.035 |         2122 |             0 |             25 |  0.987264 | 
|  3417651 | MD01.pv2.iqstack.20140824 | y.00000 | skycell.035 |         2358 |             0 |             25 |  0.987363 | 

Problems:

  • no quality faults
  • poor pixels/regions (by request)

MD02

PV2 camera & warps -- PV2 warps are not cleaned up and will also be a accessible via the PSS/PSPS interface more quickly (release name MD02.PV2).

  • label: MD02.pv2.20131101

Summary of stack produced:

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3417711 | MD02.pv2.iqstack.20140824 | g.00000 | skycell.035 |         2322 |             0 |             25 |  0.987466 | 
|  3417771 | MD02.pv2.iqstack.20140824 | r.00000 | skycell.035 |         2090 |             0 |             25 |  0.987384 | 
|  3417831 | MD02.pv2.iqstack.20140824 | i.00000 | skycell.035 |         2030 |             0 |             25 |  0.987189 | 
|  3417891 | MD02.pv2.iqstack.20140824 | z.00000 | skycell.035 |         2353 |             0 |             25 |  0.987305 | 
|  3417951 | MD02.pv2.iqstack.20140824 | y.00000 | skycell.035 |         2421 |             0 |             25 |  0.987446 | 

Problems:

  • no quality faults
  • poor pixels/regions (by request)

MD03

PV2 camera & warps -- PV2 warps are not cleaned up and will also be a accessible via the PSS/PSPS interface more quickly (release name MD03.PV2).

  • label: MD03.pv2.20131018

Summary of stack produced:

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3418011 | MD03.pv2.iqstack.20140824 | g.00000 | skycell.035 |         1954 |             0 |             25 |  0.987355 | 
|  3418071 | MD03.pv2.iqstack.20140824 | r.00000 | skycell.035 |         2147 |             0 |             25 |  0.987175 | 
|  3418131 | MD03.pv2.iqstack.20140824 | i.00000 | skycell.035 |         2189 |             0 |             25 |   0.98697 | 
|  3418191 | MD03.pv2.iqstack.20140824 | z.00000 | skycell.035 |         2250 |             0 |             25 |  0.987086 | 
|  3418251 | MD03.pv2.iqstack.20140824 | y.00000 | skycell.035 |         2454 |             0 |             25 |  0.987328 | 

Problems:

  • no quality faults
  • poor pixels/regions (by request)

MD05

PV2 camera & warps -- PV2 warps are not cleaned up and will also be a accessible via the PSS/PSPS interface more quickly (release name MD05.PV2).

  • label: MD05.pv2.20131022

Summary of stack produced:

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3418311 | MD05.pv2.iqstack.20140824 | g.00000 | skycell.035 |         2365 |             0 |             25 |  0.987169 | 
|  3418371 | MD05.pv2.iqstack.20140824 | r.00000 | skycell.035 |         2062 |             0 |             25 |  0.987095 | 
|  3418431 | MD05.pv2.iqstack.20140824 | i.00000 | skycell.035 |         2083 |             0 |             25 |  0.986385 | 
|  3418491 | MD05.pv2.iqstack.20140824 | z.00000 | skycell.035 |         2199 |             0 |             25 |  0.986534 | 
|  3418551 | MD05.pv2.iqstack.20140824 | y.00000 | skycell.035 |         2362 |             0 |             25 |  0.986763 |

Problems:

  • no quality faults
  • poor pixels/regions (by request)

MD06

PV2 camera & warps -- PV2 warps are not cleaned up and will also be a accessible via the PSS/PSPS interface more quickly (release name MD06.PV2).

  • label: MD06.pv2.20131025

Summary of stack produced:

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3418611 | MD06.pv2.iqstack.20140824 | g.00000 | skycell.035 |         2242 |             0 |             25 |  0.987401 | 
|  3418671 | MD06.pv2.iqstack.20140824 | r.00000 | skycell.035 |         2117 |             0 |             25 |  0.987385 | 
|  3418731 | MD06.pv2.iqstack.20140824 | i.00000 | skycell.035 |         2575 |             0 |             25 |  0.987218 | 
|  3418791 | MD06.pv2.iqstack.20140824 | z.00000 | skycell.035 |         2142 |             0 |             25 |  0.987234 | 
|  3418851 | MD06.pv2.iqstack.20140824 | y.00000 | skycell.035 |         2247 |             0 |             25 |  0.987404 | 

Problems:

  • quality faults -- yes, possibly shouldn't happen but like deep stacks won't be fixed for PV2
    | stack_id | label                     | filter  | skycell_id  | state | quality | reject_images | count(warp_id) | good_frac |
    +----------+---------------------------+---------+-------------+-------+---------+---------------+----------------+-----------+
    |  3418719 | MD06.pv2.iqstack.20140824 | i.00000 | skycell.021 | full  |    3012 |             0 |             11 |  0.120389 |
    
  • poor pixels/regions (by request)

MD07

PV2 camera & warps -- PV2 warps are not cleaned up and will also be a accessible via the PSS/PSPS interface more quickly (release name MD07.PV2).

  • label: MD07.pv2.20131027

Summary of stack produced:

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3418916 | MD07.pv2.iqstack.20140824 | g.00000 | skycell.035 |         2023 |             0 |             25 |  0.987248 | 
|  3418981 | MD07.pv2.iqstack.20140824 | r.00000 | skycell.035 |         2330 |             0 |             25 |  0.987178 | 
|  3419046 | MD07.pv2.iqstack.20140824 | i.00000 | skycell.035 |         2199 |             0 |             25 |  0.987095 | 
|  3419111 | MD07.pv2.iqstack.20140824 | z.00000 | skycell.035 |         2577 |             0 |             25 |  0.987147 | 
|  3419176 | MD07.pv2.iqstack.20140824 | y.00000 | skycell.035 |         2628 |             0 |             25 |  0.987274 | 

Problems:

  • quality faults -- yes, possibly shouldn't happen but like deep stacks won't be fixed for PV2
    | stack_id | label                     | filter  | skycell_id  | state | quality | reject_images | count(warp_id) | good_frac |
    +----------+---------------------------+---------+-------------+-------+---------+---------------+----------------+-----------+
    |  3418902 | MD07.pv2.iqstack.20140824 | g.00000 | skycell.017 | full  |    3012 |             0 |             13 |  0.206201 | 
    
  • quality faults because very few inputs -- unclear yet what to do about this case
    | stack_id | label                     | filter  | skycell_id  | state | quality | reject_images | count(warp_id) | good_frac |
    +----------+---------------------------+---------+-------------+-------+---------+---------------+----------------+-----------+
    |  3419018 | MD07.pv2.iqstack.20140824 | r.00000 | skycell.082 | full  |    3006 |             0 |              3 |  0.002567 | 
    |  3419091 | MD07.pv2.iqstack.20140824 | z.00000 | skycell.011 | full  |    3006 |             0 |              3 |  0.003024 | 
    |  3419148 | MD07.pv2.iqstack.20140824 | z.00000 | skycell.082 | full  |    3006 |             0 |              2 |  0.002515 | 
    |  3419156 | MD07.pv2.iqstack.20140824 | y.00000 | skycell.011 | full  |     257 |             0 |              5 |  0.044241 | 
    
  • poor pixels/regions (by request)

MD08

PV2 camera & warps -- PV2 warps are not cleaned up and will also be a accessible via the PSS/PSPS interface more quickly (release name MD08.PV2).

  • label: MD08.pv2.20131028

Summary of stack produced:

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3419236 | MD08.pv2.iqstack.20140824 | g.00000 | skycell.035 |         2107 |             0 |             25 |  0.987141 | 
|  3419296 | MD08.pv2.iqstack.20140824 | r.00000 | skycell.035 |         2140 |             0 |             25 |  0.987243 | 
|  3419356 | MD08.pv2.iqstack.20140824 | i.00000 | skycell.035 |         2439 |             0 |             25 |  0.986901 | 
|  3419416 | MD08.pv2.iqstack.20140824 | z.00000 | skycell.035 |         2310 |             0 |             25 |  0.987059 | 
|  3419476 | MD08.pv2.iqstack.20140824 | y.00000 | skycell.035 |         2639 |             0 |             25 |   0.98719 | 

Problems:

  • no quality faults
  • poor pixels/regions (by request)

MD10

PV2 camera & warps -- PV2 warps are not cleaned up and will also be a accessible via the PSS/PSPS interface more quickly (release name MD10.PV2).

  • label: MD10.pv2.20131103

Summary of stack produced:

| stack_id | label                     | filter  | skycell_id  | dtime_script | reject_images | count(warp_id) | good_frac |
+----------+---------------------------+---------+-------------+--------------+---------------+----------------+-----------+
|  3419536 | MD10.pv2.iqstack.20140824 | g.00000 | skycell.035 |         2223 |             0 |             25 |  0.987351 | 
|  3419596 | MD10.pv2.iqstack.20140824 | r.00000 | skycell.035 |         2441 |             0 |             25 |  0.987212 | 
|  3419656 | MD10.pv2.iqstack.20140824 | i.00000 | skycell.035 |         2549 |             0 |             25 |  0.986772 | 
|  3419716 | MD10.pv2.iqstack.20140824 | z.00000 | skycell.035 |         2511 |             0 |             25 |  0.986811 | 
|  3419776 | MD10.pv2.iqstack.20140824 | y.00000 | skycell.035 |         2219 |             0 |             25 |  0.987335 | 

Problems:

  • no quality faults
  • poor pixels/regions (by request)