Changes between Version 14 and Version 15 of MD.PV3

Timestamp:

Sep 24, 2014, 12:54:34 PM (12 years ago)

Author:

Mark Huber

Comment:

--

MD.PV3

@@ -27 +27 @@
 || || deep stk || 3033 || 37.5k* || 13k || TBD || *using 8 theads ||
 || || iq stk || 3325 || 2215 || 310 || TBD || edge skycells of course run faster in all stacks||
-|| || WSref diff || 2500k || ? || ? || TBD || done by season, same number as warps ||
+|| || WSref diff || 2500k || ~200 || ~50 || TBD || done by season, same number as warps ||
 || || WSdeep diff || x || x || x || x || not going to do ||
-|| || SS diff || ~291k || ? || ? || TBD || done by season, same number as night stacks ||
+|| || SS diff || ~291k || ~200 || ~50 || TBD || done by season, same number as night stacks ||
 || || night staticsky || ~291k || 110 || 40 || TBD || same number as night stacks (no filter groups) ||
 || || ref staticsky || 3025 || 4960 || 1700 || TBD || ||
@@ -40 +40 @@
 || || full-forced phot || 2500k || ? || ? || TBD || ||
 || || || || || || || ||
-|| CNP || chip/camera/warp || /2695/ || ~115/410/60 || 150/430/125 || TBD || ||
-|| || ref/deep stk || ~500? || X || x || TBD || Nin~ , seasons (first/last month for ref?)||
-|| || WSref/deep diff || ~2695 || X || X || TBD || ||
-|| || ref/deep staticsky || 
-|| || ref/deep skycal || ~ || ~37 || ~29 || TBD ||
-|| || full-forced phot || || ? || ? || TBD || ||
+|| CNP || chip/camera/warp || ~162k/2695/162k || ~115/410/60 || 150/430/125 || TBD || 2695 exposures *60 imfiles/55-60 skycells || ||
+|| || ref/deep stk || ~500? || ? || ? || TBD || Nin~ , seasons (first/last month for ref?)||
+|| || WSref/deep diff || ~2695 || ~? || ~? || TBD || ||
+|| || ref/deep staticsky || ~100? || ? || ? || TBD || ||
+|| || ref/deep skycal || ~500? || ~37? || ~29? || TBD ||
+|| || full-forced phot || ~162k || ? || ? || TBD || ||
 || || || || || || || ||
-|| M31 || chip/camera/warp || /4347/ || 129/624/62 || 190/313/30 || TBD || 
+|| M31 || chip/camera/warp || ~261/4347/261k || 129/624/62 || 190/313/30 || TBD || 4347 exposures *60 imfiles/55-60 skycells  || || 
 || || background restore ||
 || || ref stk || ~500? || X || x || TBD || Nin~?, 4 seasons ||
@@ -56 +56 @@
 || || full-forced phot ||  || ? || ? || TBD || ||
 || || || || || || || || 
-|| STS || chip/camera/warp || /16606/ || 210/1200/100 || 120/575/70 || TBD || Nin~?, 4 seasons||
-|| || ref stk || ~3500? || X || x || TBD || Nin~ ||
-|| || deep stk || ? 
+|| STS || chip/camera/warp || ~996k/16606/996k || 210/1200/100 || 120/575/70 || TBD || 16606 exposures *60 imfiles/55-60 skycells || ||
+|| || ref stk || ~3500? || X || x || TBD || Nin~, 4 seasons ||
+|| || deep stk || ? ||
 || || WSref diff || ~ || X || X || TBD || ||
 || || ref staticsky ||
@@ -73 +73 @@
  * MD: will be run one field at a time to limit the queued jobs
   * chip/cam/warp ~6 jobs/node ~90 jobs, ~42+3+27 ~72 days
-  * iq stack ~4 jobs/node ~60 jobs, 
-  * night stack ~4 jobs/node ~60 jobs, 
-  * ref stack ~3 jobs/node ~45 jobs, 
+  * night stack ~4 jobs/node ~60 jobs, ~48.2 d
+  * ref stack ~4 jobs/node ~60 jobs, ~19.5 d (pv2 took ~2 weeks but also used extra nodes)
   * deep stack ~1 jobs/node ~15 jobs, ~88 days (pv2 took ~70d, may take longer for pv3 if don't let as much get clipped for Nin on edge skycells)
-  * WSref diff ~4 jobs/node 60 jobs
-  * SSref diff ~4 jobs/node 60 jobs
-  * all staticsky + skycal ~4 and 8 jobs/node 60 and 120 jobs
-  * total ~
+  * iq stack ~4 jobs/node ~60 jobs, ~1.4 d
+  * WSref diff ~5 jobs/node 75 jobs, ~77.1 d
+  * SSref diff ~5 jobs/node 75 jobs, ~9 d
+  * all staticsky + skycal ~4 and 8 jobs/node 60 and 120 jobs, ~10.4, 0.9 d
+  * total ~ 333.1 d, ~worst case but >60 dedicated processing nodes will likely be needed to get done w/in 3 months
  * CNP:
  * M31:
@@ -89 +89 @@
  * disk space limiting
  * chip-warp:
-  * refcat oddities in field
+  * check if refcat oddities in field problem here
   * background restore M31 and masking for M31/STS -- test samples to be run
  * stack  
   * stack quality fault
-  * night stack config, refstack edge loss
-  * ref+deep stack input cuts
-  * iq+ref+deep test stacks for CNP,M31,STS 
+  * night stack config, refstack edge loss -- see diffim as well
+  * ref+deep stack input cuts -- need feedback on if 2009 and or early 2010 data is okay to use
+  * iq+ref+deep test stacks for CNP,M31,STS and to verify Nin and input cuts
  * staticsky/skycal
-  * zeropt recalib in skycal 
   * stack photometry sky oddities
+  * check refcat oddities in the MD fields
+  * zeropt recalib in skycal improvement and characterization (y-band at least needs own config)
  * diffim
   * verify similar photometry