Summary of MD04.pv2_stacktest.20130903

(back to MediumDeepFields summary)

This is a test sample for PV2 and deep stacks using MD04 while it is an inactive field.

PV2: update type reprocessing

Deep stacks:

• for comparison, the last deep stack test using MD09 PV1 (MD09.PV1_deepstack_tests)

Reference stack -- this should be useful for making the MD04 refstack for return to field early November 2013. May move section for refstack to its own page for consistency.

Other stacks:

• in and out of season stacks -- haven't been done yet, need sample for Taiwan meeting
• fewer N input refstack

Status:

• prior 9/3 many issues being worked out for PV2 and MD (MD10.refstack.20130715)
• 9/3 PV2 updates started for MD04 after M31 reprocessing finished
• 9/6 PV2 updates finished
• 10/3 test all-in deep stack skycell run
• 10/10 started full all-in deep stack running
• 11/03 deep stack test with cuts started -- deep stack tests having problems
• 11/22 (great amount of updating to do)
• 11/26 all stack and staticsky should be finished and available on datastore now -- all stack types should have label and data_group to indicate what they are

---

PV2 updates

MD04

PV1 camera label -- data_group: mix of processing labels/data_groups, but all of PV1 type (cam_id>=434615)

label	data_group
MD04.pv1.20130322	MD04.pv1.20130322
MD04.refstack.20121125	MD04.refstack.20121125
MD04.nightlyscience	MD04.<obsdate 20121129--20130525>

There have been some test reprocessing that had to be avoided -- czw.MDVDtest%, czw.MDVD.MD04.%

PV2 camera (as well as chip, warp) label MD04.pv2.20130903, data_group left to what originally set as in order to be able to track backwards to the processing group.

With ipp047 data disk down, have found ~3% of exposures missing a related imfile (~half are the burn.tbl files that can be regenerated) and indicates serious need of fixing the replication of raw data. Also found some replicated but on a host (ipp064X) that no longer exists.. it is also possible some replicated products may be on the same host if push to the stsci nodes with 3 partitions..

Number totals: (not all exposures pass camera stage, FWHM>12 pixels)

| filter  | count(cam_id) | count(warp_id) |
+---------+---------------+----------------+
| g.00000 |           596 |            577 | 
| i.00000 |           719 |            713 | 
| r.00000 |           588 |            571 | 
| y.00000 |           471 |            465 | 
| z.00000 |           711 |            710 | 

PV2 warps will also be kept available (not cleaned up) for making deep stacks but also will be a accessible via the PSS/PSPS interface more quickly (release name MD04.PV2).

3PI overlap

Unclear what want to do here still, with LAP running, and how useful it will be in the stack -- like MD10.refstack.20130715, will make a sample refstack with 3PI addition

---

Deep Stacks

Like with MD09, different versions of stacks will be produced for testing (not all will be full-field?)

• all --
• moderate input cuts --
• best cuts >2010 and moderate input cuts --

All

Basically all warps available with ~FWHM <13 pixels to avoid any badly measured PSF warps.

• stack -- label, data_group = MD04.alldeeptest.20131010
• staticsky -- label, data_group = MD04.alldeeptest.20131010.Nx20131122, MD04.alldeeptest.20131010.<#>x20131122 -- where <#> is the number of filters available
• skycal -- label, data_group = MD04.alldeeptest.20131010.Nx20131122.cal20131122, MD04.alldeeptest.20131010.Nx20131122.cal20131122

Cut

All warps FWHM <10 pixels and input warp cuts

• ZPT > ZPTmean-0.33

  g: 24.11
  r: 24.32
  i: 24.07
  z: 23.81
  y: 22.79
  

• elongation <1.5
• background:

  g: <400
  r: <500
  i: <1800
  z: <2500
  y: <5000
  

• stack -- label, data_group = MD04.cutdeeptest.20131103
• staticsky -- label, data_group = MD04.cutdeeptest.20131103.Nx20131122, MD04.cutdeeptest.20131103.<#>x20131122 -- where <#> is the number of filters available
• skycal -- label, data_group = MD04.cutdeeptest.20131103.Nx20131122.cal20131126, MD04.cutdeeptest.20131103.Nx20131122.cal20131126

Best Cut

All warps FWHM <10 pixels and the same input warp cuts, only season years 2-4 (excludes 2009/2010 season)

• stack -- label, data_group = MD04.cut2009deeptest.20131103
• staticsky -- label, data_group = MD04.cut2009deeptest.20131103.Nx20131122, MD04.cut2009deeptest.20131103.<#>x20131122 -- where <#> is the number of filters available
• skycal -- label, data_group = MD04.cut2009deeptest.20131103.Nx20131122.cal20131122, MD04.cut2009deeptest.20131103.Nx20131122.cal20131122

---

Reference Stacks

In addition to the normal reference stack for a season, test samples of planned final reference stack types also being made.

| exp_name    | filter  | dateobs             | comment                         |
+-------------+---------+---------------------+---------------------------------+
| o4777g0224o | g.00000 | 2008-11-07 14:56:59 | MD04 g MJD54777 MD04 center     | 
..
| o4803g0354o | r.00000 | 2008-12-03 15:25:16 | MD04 r MJD54803 MD04 center     | 
---- pre-survey

| o5161g0222o | g.00000 | 2009-11-26 13:55:38 | MD04 g,r MJD55161 g MD04 center | 
..
| o5321g0014o | i.00000 | 2010-05-05 06:20:20 | MD04 i N5321 MD04 center        | 
---- yr1 -- 11/26/2009 -- 5/5/2010

| o5545g0436o | g.00000 | 2010-12-15 12:12:35 | MD04 g,r N5545 g MD04 center    | 
..
| o5681g0089o | i.00000 | 2011-04-30 07:34:50 | MD04 i N5681 MD04 center        | 
---- yr2 -- 12/15/2010 -- 4/30/2011

| o5892g0155o | g.00000 | 2011-11-27 14:41:57 | MD04 g,r N5892 g MD04 center    | 
..
| o6056g0114o | z.00000 | 2012-05-09 07:38:31 | MD04 z N6056 MD04 center        | 
---- yr3 -- 11/27/2011 -- 5/09/2012

| o6260g0448o | y.00000 | 2012-11-29 15:14:41 | MD04 y N6260 MD04 center        | 
..
| o6437g0061o | y.00000 | 2013-05-25 05:50:54 | MD04 y N6437 MD04 center        | 
---- yr4 -- 11/29/2012 -- 5/25 2013

---- yr5 -- actively being observed since 2013-11-08

(more details to follow)

Basic Reference Stack

Default reference stacks for nightly diffims

• reduction MD_REF_STACK_1DG, FWHM <6 pixel and 1D convolution
• best ~60 input warps excluding 2009/2010 year/season
• stack -- label, data_group = MD04.refstack.20131018, MD04.refstack.20131018_1dg -- _1dg to indicate 1D convolution method, but is now default for all stacks

• staticsky -- label, data_group = MD04.refstack.20131018.Nx20131122, MD04.refstack.20131018.<F>x20131122 -- where <F> is the number of filters available for the skycell
• skycal -- label, data_group = MD04.refstack.20131018.Nx20131122.cal20131122, MD04.refstack.20131018.Nx20131122.cal20131122

IQ Stack

Basically a best reference stack possible sample -- 10-15 inputs, uses MD_DEEP_STACK_1DG so no extra cuts on FWHM made in order to look for bad warps need to be excluded in future runs

• stack -- label, data_group = MD04.iqtest.20131118
• staticsky -- label, data_group = MD04.iqtest.20131118.Nx20131122, MD04.iqtest.20131118.<#>x20131122 -- where <#> is the number of filters available
• skycal -- label, data_group = MD04.iqtest.20131118.Nx20131122.cal20131122, MD04.iqtest.20131118.Nx20131122.cal20131122

Reference Yearly/Seasonal Stack

Basically a best reference stack possible using the specific year/season data -- 30-35 warp inputs, used MD_DEEP_STACK_1DG so no extra cuts on FWHM made in order to look for bad warps need to be excluded in future runs

• stack -- label, data_group = MD04.yr<#>.reftest.20131118, where <#> = 1-4
• staticsky -- label, data_group = MD04.yr<Y>.reftest.20131118.Nx20131122, MD04.yr<YY>.reftest.20131118.<F>x20131122 -- where <Y> is the year/season, <F> is the number of filters available
• skycal -- label, data_group = MD04.yr<Y>.reftest.20131118.Nx20131122.cal20131126, MD04.yr<Y>.reftest.20131118.Nx20131122.cal20131126 -- where <Y> is the year/season

Comparison of yr1 and yr2-4 may help show if 2009/2010 season should be excluded from other stack types due to baffle improvements.

Will need to look for any target events lost due to fewer inputs (field edge events) and compare to the MD04.iqtest.20131118.

Reference Out-of-Season Stack

Two types possible, all exposures before season (default during active observing) and all exposures before/after season (or N seasons)

Not currently possible with current tools unless add warp_id manually for each skycell -- this method may even be necessary for all final stacks to exclude problematic exposure/chips in a warp skycell

Sample test for not season 2 and 3, only including warps from season 1 (2009/2010) and 4 (2012/2013)

• stack -- label, data_group = MD04.notyr2_3.reftest.20131124
• staticsky -- label, data_group = MD04.notyr2_3.reftest.20131124.Nx20131124, MD04.notyr2_3.reftest.20131124.<F>x20131124 -- where <F> is the number of filters available
• skycal -- label, data_group = MD04.notyr2_3.reftest.20131124.Nx20131124.cal20131126, MD04.notyr2_3.reftest.20131124.Nx20131124.cal20131126