IPP Progress Report for the week 2011.08.29 - 2011.09.02

(Up to IPP Progress Reports)

Eugene Magnier

  • finished major development work on psphotStack and iterative kron mags for galaxies
  • finished re-rsyncing the data partitions for ipp010, ipp017, ipp020 -- all wave up refurb work is done except for the ipp021 mobo upgrade

Serge Chastel

Heather Flewelling

  • Zoology investigations (see http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/Zoology)
  • Answered Roy's LAP dvodb questions (there are many missing chips in the camera stage, this has to do with how the diffs are created, no easy fix)
  • grabbed files for JT
  • grabbed files for TD
  • had meeting with the Erics - will help Eric get started on IPP/skyprobe once ifaps1 is back up
  • rsync'd all catdirs from ipp006 to ipp004
  • czar Friday

Roy Henderson

  • ippToPsps
    • investigated LAP exposures not found in DVO: destreak issue
    • added 'deletion policy' section to the config file: local disk, datastore and dxlayer files all have to be deleted at some point
    • added method to check if a batch has consistently failed, so it can be skipped form future queing
    • changes to code to poll ODM: logging load failures, grabbing the ODM comment so that I can investigate and re-submit
    • further investigations into 0 instFlux detections
    • wrote a MySQL stored procedure for erfc() function for likelihood calculations. Huge improvement to speed (had been using a cursor)
  • czar mon/tues
    • fixed IPP metrics, which were broken when we switched ippMonitor over from ipp004
    • finally fixed issue of metrics not getting automatically updated
  • Tiger team
    • cuts program now produces ds9 region files per skycell per filter
    • cuts program now produces histograms showing cuts per filter
    • work on sloan cross-matching:
      • imported sloan stripe 82 into our database
      • used TOPCAT in an attempt to reproduce the Durham cross-matching results. Successful, after realizing that Stripe 82 does not cover the whole SAS footprint
      • plots for different cross-matching errors

Mark Huber

  • MD02.V3 refstack processing started. MD10, 01 evaluation continues.
  • Increased difference image detections in some exposures look to be caused by glitch from tracking or aberration. Looking into condition to identify and flag in future images.
  • simtest IPP->PSPS test configuration debugging continued.

Bill Sweeney

  • Wrote new implementation for psLib to read fits tables in a memory efficient way. Tested with ppMops and streaksremove. Memory usage reduced ~90%.
  • Investigated large number of destreak failures for camera stage. It turned out to be triggered by change to magic for lap that is inconsistent with the design of the system. Punted problem back to Chris.
  • Out two days with bad back.

Chris Waters

  • LAP: Identified problem in magicRun queuing due to LAP having multiple diffs from a single exposure with a given label. Created workaround that seems to be able to keep data moving. Began work to completely solve this problem, along with a related bug preventing warp-stack diffs from being fully included in DVO. Continued queuing data.
  • Diskspace: Analysed disk usage script results and concluded that we seem to have a stable processing buffer, but are adding to the final result usage at a faster than expected rate. Modified these scripts to run via cronjob, so we can have weekly statistics about the current disk usage.
  • Processing: Changed cshrc configuration to select a nebulous server from a list automatically. This should prevent the loss of one of these servers from dragging processing down.