IPP Progress Report for the week 2011.08.01 - 2011.08.05

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• MOPS czar
• mysql replication issues because of ippdb03 crash
• Modified replication scripts to save master coordinates
• Updated associated documentation
• NFS check is now running on ippdb0[0-3]
• Reviewed PSPS Test Plan for Conrad
• http+nfs stopped on neverland (were they needed?)

Heather Flewelling

• vacation of awesomeness

Roy Henderson

• DVO duplicates investigation:
  • some confusion over DVO IMAGE_ID-EXTERN_ID relationship. Finally got code making sense
  • added nice, safe code to re-create DVO master tables in MySQL
  • ran code, found duplicates, and showed Gene. Appears to be a chip-boundary issue

• ippToPsps coding:
  • all programs now take a single argument: a path to a config
  • rewrote convertPhotCodesToXml.pl to produce VOTable-compliant output
  • a lot of awk scripting to convert old IN data format to VOTable
  • main ippToPsps program can now produce and publish IN batches. Old C and Perl code has been purged. Very satisfying.
  • extra methods for logging class, including some debug messages and a 'title' method
  • now using one, and only one, name for batch types: P2, ST, IN. Changed directory structure for configs accordingly
  • added batch types to config
  • new section for config enabling a list of IDs (cam_id or stack_id) to process (useful for testing)

• documentation
  • lots of new comments for ippToPsps
  • a lot of new documentation for ippToPsps on the wiki. Pretty much up-to-date now

• investigated crazy high fluxes seen in last load to PSPS:
  • changed code to enable processing of a specific exposure (or exposures)
  • found smfs with crazy PSF_INST_MAG values (i.e. < -20)
  • implemented a cut for PSF_INST_MAG < -17.5, as recommended by Gene

Mark Huber

• Additional testing of ppSub and convolution direction for MD10.V3 stack-stack diffs. Prep for new MD01.V3 tessellation.
• simtest IPP->PSPS: still working on the configuration. Changed from original simtest field to MD04.
• Czar Friday.

Bill Sweeney

Chris Waters

• DQStats: Finally fixed the SQL to only publish new results, but to correctly publish all new results.
• Refcat zeropoints: Reprocessed SAS2 z-filter data as part of testing of the new LAP code, using the synthetic catalog instead of the new reference catalog. This eliminates the position dependent zeropoint offsets that were observed, pointing to issues in the reference catalog in those regions.
• LAP: Further coding and debugging of LAP code to eliminate quickstacks. Once final tests are complete, a new set of LAP runs will be queued.
• ROC: Continued correcting these files to match real disk files. It appears (from a not entirely unbiased sample) that the majority of missing imfiles can either be reconstructed from N-th copies still on disk, or that enough other files in a given ROC can be recovered to fix individual missing files. No statistics are complete on this, and more work needs to be done to allow this to run automatically.
• Out one day due to illness and the inadvertent slaughter of my laptop by a glass of water.