IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 43 and Version 44 of MD.PV3


Ignore:
Timestamp:
Jun 4, 2015, 2:18:36 PM (11 years ago)
Author:
Mark Huber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • MD.PV3

    v43 v44  
    3030 * 4/01 -- setting up using UH HPC for seasonal refstacks, then night stacks  -- refstacks running
    3131 * 4/26 -- UH HPC now dominated by 3PI-full forced jobs -- no nodes available
     32 * night stacks processing on local machines -- problem with excess rejection and edge loses due to poor warp PSF models, will need development/code work to fix
     33 * 6/04 -- night stack ~50% finished in bulk, still problem with excess edge loss and fields not complete until fixed -- timeframe estimated end of June with ref stack verification
    3234 
    3335
     
    206208
    207209Issues to address, most before starting:
    208  * disk space limits (turned out to also include network limits) --
    209  * config reorganization or specific PV3 named ones to create -- all configs must have PV3 to avoid confusion
     210 * disk space limits (turned out to also include network limits) -- mostly solved
     211 * config reorganization or specific PV3 named ones to create -- all configs must have PV3 to avoid confusion -- IQ stack, WS+SS diff, staticsky+skycal, FF setup remaining
    210212 * chip-camera-warp:
    211   * check if refcat oddities in field is a problem -- Eddie and Doug catalogs to compare to
    212   * new refcat astrometry and zeropoint verification tests -- the new refcat demonstrated large astrometry and zeropoint variations when previous config used
    213   * CNP tessellation check and exposure filling factor
    214   * MD as a targeted survey and utilized throughout the different processing needs to have a summary of exposures changed/lost in PV3 -- PV3 vs PV2 and original processing comparison in quality faults and astrometry/zeropoint variations is critical so events published in transients work is reproduced
    215   * background restore M31 and masking for M31/STS -- test samples to be run and verified for final version
     213  * check if refcat oddities in field is a problem -- Eddie and Doug catalogs to compare to (TBD)
     214  * new refcat astrometry and zeropoint verification tests -- the new refcat demonstrated large astrometry and zeropoint variations when previous config used -- final verification needed with full fields finished
     215  * CNP tessellation check and exposure filling factor -- done, need plots for notes wiki
     216  * MD as a targeted survey and utilized throughout the different processing needs to have a summary of exposures changed/lost in PV3 -- PV3 vs PV2 and original processing comparison in quality faults and astrometry/zeropoint variations is critical so events published in transients work is reproduced -- TBD before final night+ref stack and diffs done
     217  * background restore M31 and masking for M31/STS -- test samples to be run and verified for final version, on hold for now 1/2015
    216218 * stack 
    217   * stack kernel files -- for deep stacks can be >400 and any one can cause a fault in the middle of processing if system is unstable.. for deep stacks may be helpful to disable those since not doing anything with conv products and shouldn't clean up deep stacks..
     219  * stack kernel files -- for deep stacks can be >400 and any one can cause a fault in the middle of processing if system is unstable.. for deep stacks may be helpful to disable those since not doing anything with conv products and shouldn't clean up deep stacks.. -- requires devel/code work and lower prio
    218220  * stack quality fault -- sometime edge skycells fault w/ internal photometry/PSF error on the convolved stack and do not want to lose for staticsky or diffims --
    219   * night stack config, refstack edge loss -- see diffim for as well -- comparison to PV2 and earlier datasets
    220   * reference, deep stack input cuts -- need feedback on if 2009 and or early 2010 data is okay to use -- 2009 is not great data so not to be used, Jan/June 2010 is still under consideration if the baffle is critical or not.
     221  * night stack config, refstack edge loss -- see diffim for as well -- comparison to PV2 and earlier datasets -- TBD w/ final night+ref stacks
     222  * reference, deep stack input cuts -- need feedback on if 2009 and or early 2010 data is okay to use -- 2009 is not great data so not to be used, Jan/June 2010 seems okay (level of impact unknown) and has been used in deep stack
    221223  * iq, reference, deep test stacks for CNP (M31,STS) and to verify Nin, input cuts, pixel rejection and minimum pixel limits --
    222224 * staticsky/skycal
    223   * stack photometry sky oddities --
    224   * check refcat oddities in the MD fields for skycal re-zeropointing
    225   * zeropt recalibration in skycal improvement and characterization (y-band at least may need own config) --
    226   * evaluate test samples for CNP (M31,STS) -- equivalent stacks in available filters vs deeper in primary filter, single filter only for STS/M31 so no extended source fits?
     225  * stack photometry sky oddities --  
     226  * check refcat oddities in the MD fields for skycal re-zeropointing -- TBD
     227  * zeropt recalibration in skycal improvement and characterization (y-band at least may need own config) -- TBD
     228  * evaluate test samples for CNP (M31,STS) -- equivalent stacks in available filters vs deeper in primary filter, single filter only for STS/M31 so no extended source fits? -- TBD
    227229 * diffim
    228   * verify similar photometry to previous reductions --
    229   * verify similar detections/coverage (no sig loss on edges) -- a key point for reference stack creation above
    230   * evaluate test samples for CNP (M31,STS) --
    231  * full-forced photometry -- unknown, untested still for any MD type field
     230  * verify similar photometry to previous reductions -- TBD
     231  * verify similar detections/coverage (no sig loss on edges) -- a key point for reference stack creation above -- TBD
     232  * evaluate test samples for CNP (M31,STS) -- TBD
     233 * full-forced photometry -- unknown, untested still for any MD type field -- TBD
    232234
    233235----
     
    261263  * MD09 --
    262264
    263  * seasonal reference -- tag ipp-mdpv3-20150326
    264 
    265  * night -- tag ?
     265 * seasonal reference -- tag ipp-mdpv3-20150326 on UH HPC cray system
     266
     267 * night -- tag ipp-mdpv3-20150326 on default IPP/Maui systems
    266268
    267269 * iq -- tag ?
     
    277279(should have own subsections for adding summary details on config/processing changes)
    278280
    279  * WSdiff -- cannot start until seasonal reference stacks finish
    280 
    281  * SSdiff -- cannot start until night and seasonal reference stacks finish
     281 * WSdiff -- cannot start until seasonal reference stacks finish and verified
     282
     283 * SSdiff -- cannot start until night and seasonal reference stacks finish and verified
    282284
    283285----