IPP Progress Report for the week 2012.12.10 - 2012.12.16

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

  • IPP Pantasks/Condor documentation
  • Updated (and deployed) the files monitoring software. Two instances shouldn't be running at the same time now. Documentation here: FilesMonitoring
  • Vacation: Wed->Fri (until 2013-01-07)

Heather Flewelling

  • kepler
    • made minidvos of cam and skycal for kepler
    • merged minidvos
  • q3
    • merging three pi minis to q3 - still in progress, feels slow (?)
    • set to wait about half because they have a defective Ps1_v4 (gene understands this, he wants to write a way to fix that)
    • investigated the failed 3pi mini - failed because it was a transitional minidvodb - started off with one Ps1_v4, and it was modified half way through. don't do that.
  • SSP
    • verified ipp can now access otis1 (prep for ssp auto-download)
    • sent files to Steph
  • answered and investigated piles of problems with sa10, see Sa10Problems wiki
  • various fixes to ipptopsps for SA12
  • staged files for Daniel
  • staged grabs of various db columns for Jean-Charles
  • attempted to answer Mike's questions about ps1

Mark Huber

  • MD.GR0 -- MD01 finished, checked, distributed remaining V3 nightly stack processing (wiki not updated yet). Prep'd MD02,03
  • MD WSdiffim debugging MD_WSdiffim_photcheck_test20121004 -- AP_MAG differences appear connected to background superpixels, ppSub PSF appears unstable
  • czar

Bill Sweeney

This progress report covers the past two weeks

  • Most of time spent monitoring the progress and debugging the LAP static sky processing.
    • Regions completed
      • 267 < RA < 303 DEC > 10
      • RA < 225 DEC < 10
    • Debugged and fixed a couple of assertion failures in psphot.
    • Changed scripts to collect statistics on number of detections and added them to the staticskyResult table'
    • removed "singlefilter" mode of staticsky. For consistency it now always uses psphotStack even if there is only a single filter
    • investigated instances of large numbers of forced detections caused by run away y band stacks. Created a method to identify the problem and in those cases turn off matching of all y band detections. Affects about ~2% of skycells. Made a mistake in the implementation and so had to re-run 3 days worth of skycells.
    • spent seemingly vast amounts of time choosing the right balance of nodes to use for staticsky. 3 x compute 3 works well at high galactic latitudes but must dedicate an entire machine near the plane. Near the galactic center the footprint analysis is very inefficient. There are so many sources that they get merged into just a few and it takes hours for the code to figure that out.
  • Enhanced cleanup process
    • log progress to a proper log file. This will aid debugging of problems and help reduce the disk space usage in ~ipp/cleanup/pantasks_logs
    • added mode to cleanup where it will go back and insure that previously cleaned components are all cleaned up. This will be used to go back and delete the unneeded chip and warp stage cmf files which are using considerable space on the cluster
    • handle several error conditions due to lost files. Re-institute setting of error_cleaned. Combined with log files this will allow us to better identify data that has been lost due to disk failures.
  • debugged some problems with dvo at MPIA. Turned out to be an incomplete copy of the dvo catalog.
  • PSPS related issues
    • fixed setting of Object.xStackDetectID in ippToPsps, correctly this time
    • implemented Gene's fix to bug in setting raErr and decErr
    • commented on some of the issues raised on the SAS 11 issues wiki page
    • Investigated user's question as to why PSPS Objects contain detections from multiple DVO objects.

Chris Waters

  • Finished stack background combination code, and merged it into trunk. Likely need to update it to handle the zeropoint issues identified on Friday. Ran test case for David Thilker and posted results at Background_Regeneration.
  • Ran crosstalk tests for MOPS. The issue seems to be that some crosstalk positions create ghosts that are somewhat brighter than others. This causes some objects to not be flagged as crosstalk artifacts.
  • MOPS meeting. Will wait for WS diffs to be automated until MOPS is ready to handle them as well.