Summary of MD04.pv2_stacktest.20130903

(back to MediumDeepFields summary)

This is a test sample for PV2 and deep stacks using MD04 while it is an inactive field.

PV2: update type reprocessing

Deep stacks:

• for comparison, the last deep stack test using MD09 PV1 (MD09.PV1_deepstack_tests)

Reference stack -- this should be useful for making the MD04 refstack for return to field early November 2013. May move section for refstack to its own page for consistency.

Other stacks:

• in and out of season stacks -- haven't been done yet, need sample for Taiwan meeting
• fewer N input refstack

Status:

• prior 9/3 many issues being worked out for PV2 and MD (MD10.refstack.20130715)
• 9/3 PV2 updates started for MD04 after M31 reprocessing finished
• 9/6 PV2 updates finished
• 10/3 test all-in deep stack skycell run
• 10/10 started full all-in deep stack running
• 11/03 deep stack test with cuts started -- deep stack tests having problems
• 11/22 (great amount of updating to do)
• 11/26 all stack and staticsky should be finished and available on datastore now -- all stack types should have label and data_group to indicate what they are

---

PV2 updates

MD04

PV1 camera label -- data_group: mix of processing labels/data_groups, but all of PV1 type (cam_id>=434615)

label	data_group
MD04.pv1.20130322	MD04.pv1.20130322
MD04.refstack.20121125	MD04.refstack.20121125
MD04.nightlyscience	MD04.<obsdate 20121129--20130525>

There have been some test reprocessing that had to be avoided -- czw.MDVDtest%, czw.MDVD.MD04.%

PV2 camera (as well as chip, warp) label MD04.pv2.20130903, data_group left to what originally set as in order to be able to track backwards to the processing group.

With ipp047 data disk down, have found ~3% of exposures missing a related imfile (~half are the burn.tbl files that can be regenerated) and indicates serious need of fixing the replication of raw data. Also found some replicated but on a host (ipp064X) that no longer exists.. it is also possible some replicated products may be on the same host if push to the stsci nodes with 3 partitions..

Number totals: (not all exposures pass camera stage, FWHM>12 pixels)

| filter  | count(cam_id) | count(warp_id) |
+---------+---------------+----------------+
| g.00000 |           596 |            577 | 
| i.00000 |           719 |            713 | 
| r.00000 |           588 |            571 | 
| y.00000 |           471 |            465 | 
| z.00000 |           711 |            710 | 

PV2 warps will also be kept available (not cleaned up) for making deep stacks but also will be a accessible via the PSS/PSPS interface more quickly (release name MD04.PV2).

3PI overlap

Unclear what want to do here still, with LAP running, and how useful it will be in the stack -- like MD10.refstack.20130715, will make a sample refstack with 3PI addition

---

Deep Stacks

Like with MD09, different versions of stacks will be produced for testing (not all will be full-field?)

• all --
• moderate input cuts --
• best cuts >2010 and moderate input cuts --

All

Basically all warps available with ~FWHM <13 pixels to avoid any badly measured PSF warps.

• stack -- label, data_group = MD04.alldeeptest.20131010
• staticsky -- label, data_group = MD04.alldeeptest.20131010.Nx20131122, MD04.alldeeptest.20131010.<#>x20131122 -- where <#> is the number of filters available
• skycal -- label, data_group = MD04.alldeeptest.20131010.Nx20131122.cal20131122, MD04.alldeeptest.20131010.Nx20131122.cal20131122

Cut

All warps FWHM <10 pixels and input warp cuts

• ZPT > ZPTmean-0.33

  g: 24.11
  r: 24.32
  i: 24.07
  z: 23.81
  y: 22.79
  

• elongation <1.5
• background:

  g: <400
  r: <500
  i: <1800
  z: <2500
  y: <5000
  

• stack -- label, data_group = MD04.cutdeeptest.20131103
• staticsky -- label, data_group = MD04.cutdeeptest.20131103.Nx20131122, MD04.cutdeeptest.20131103.<#>x20131122 -- where <#> is the number of filters available
• skycal -- label, data_group = MD04.cutdeeptest.20131103.Nx20131122.cal20131126, MD04.cutdeeptest.20131103.Nx20131122.cal20131126

Best Cut

All warps FWHM <10 pixels and the same input warp cuts, only season years 2-4 (excludes 2009/2010 season)

• stack -- label, data_group = MD04.cut2009deeptest.20131103
• staticsky -- label, data_group = MD04.cut2009deeptest.20131103.Nx20131122, MD04.cut2009deeptest.20131103.<#>x20131122 -- where <#> is the number of filters available
• skycal -- label, data_group = MD04.cut2009deeptest.20131103.Nx20131122.cal20131122, MD04.cut2009deeptest.20131103.Nx20131122.cal20131122

---

Reference Stacks

In addition to the normal reference stack for a season, test samples of planned final reference stack types also being made.

| exp_name    | filter  | dateobs             | comment                         |
+-------------+---------+---------------------+---------------------------------+
| o4777g0224o | g.00000 | 2008-11-07 14:56:59 | MD04 g MJD54777 MD04 center     | 
..
| o4803g0354o | r.00000 | 2008-12-03 15:25:16 | MD04 r MJD54803 MD04 center     | 
---- pre-survey

| o5161g0222o | g.00000 | 2009-11-26 13:55:38 | MD04 g,r MJD55161 g MD04 center | 
..
| o5321g0014o | i.00000 | 2010-05-05 06:20:20 | MD04 i N5321 MD04 center        | 
---- yr1 -- 11/26/2009 -- 5/5/2010

| o5545g0436o | g.00000 | 2010-12-15 12:12:35 | MD04 g,r N5545 g MD04 center    | 
..
| o5681g0089o | i.00000 | 2011-04-30 07:34:50 | MD04 i N5681 MD04 center        | 
---- yr2 -- 12/15/2010 -- 4/30/2011

| o5892g0155o | g.00000 | 2011-11-27 14:41:57 | MD04 g,r N5892 g MD04 center    | 
..
| o6056g0114o | z.00000 | 2012-05-09 07:38:31 | MD04 z N6056 MD04 center        | 
---- yr3 -- 11/27/2011 -- 5/09/2012

| o6260g0448o | y.00000 | 2012-11-29 15:14:41 | MD04 y N6260 MD04 center        | 
..
| o6437g0061o | y.00000 | 2013-05-25 05:50:54 | MD04 y N6437 MD04 center        | 
---- yr4 -- 11/29/2012 -- 5/25 2013

---- yr5 -- actively being observed since 2013-11-08

(more details to follow)

Basic Reference Stack

Default reference stacks for nightly diffims

• reduction MD_REF_STACK_1DG, FWHM <6 pixel and 1D convolution
• best ~60 input warps excluding 2009/2010 year/season
• stack -- label, data_group = MD04.refstack.20131018, MD04.refstack.20131018_1dg -- _1dg to indicate 1D convolution method, but is now default for all stacks

• staticsky -- label, data_group = MD04.refstack.20131018.Nx20131122, MD04.refstack.20131018.<F>x20131122 -- where <F> is the number of filters available for the skycell
• skycal -- label, data_group = MD04.refstack.20131018.Nx20131122.cal20131122, MD04.refstack.20131018.Nx20131122.cal20131122

IQ Stack

Basically a best reference stack possible -- used MD_DEEP_STACK_1DG so no extra cuts on FWHM made in order to look for bad warps need to be excluded in future runs

• stack -- label, data_group = MD04.iqtest.20131118
• staticsky -- label, data_group = MD04.iqtest.20131118.Nx20131122, MD04.iqtest.20131118.<#>x20131122 -- where <#> is the number of filters available
• skycal -- label, data_group = MD04.iqtest.20131118.Nx20131122.cal20131122, MD04.iqtest.20131118.Nx20131122.cal20131122

Reference Yearly/Seasonal Stack

Basically a best reference stack possible using the specific year/season data -- used MD_DEEP_STACK_1DG so no extra cuts on FWHM made in order to look for bad warps need to be excluded in future runs

• stack -- label, data_group = MD04.yr<#>.reftest.20131118, where <#> = 1-4
• staticsky -- label, data_group = MD04.yr<Y>.reftest.20131118.Nx20131122, MD04.yr<YY>.reftest.20131118.<F>x20131122 -- where <Y> is the year/season, <F> is the number of filters available
• skycal -- label, data_group = MD04.yr<Y>.reftest.20131118.Nx20131122.cal20131126, MD04.yr<Y>.reftest.20131118.Nx20131122.cal20131126 -- where <Y> is the year/season

Comparison of yr1 and yr2-4 may help show if 2009/2010 season should be excluded from other stack types due to baffle improvements.

Reference Out-of-Season Stack

Two types possible, all exposures before season (default during active observing) and all exposures before/after season (or N seasons)

Not currently possible with current tools unless add warp_id manually for each skycell -- this method may even be necessary for all final stacks to exclude problematic exposure/chips in a warp skycell

Sample test for not season 2 and 3, only including warps from season 1 (2009/2010) and 4 (2012/2013)

• stack -- label, data_group = MD04.notyr2_3.reftest.20131124
• staticsky -- label, data_group = MD04.notyr2_3.reftest.20131124.Nx20131124, MD04.notyr2_3.reftest.20131124.<F>x20131124 -- where <F> is the number of filters available
• skycal -- label, data_group = MD04.notyr2_3.reftest.20131124.Nx20131124.cal20131126, MD04.notyr2_3.reftest.20131124.Nx20131124.cal20131126