IPP Progress Report for the week 2011.07.18 - 2011.07.22

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• MOPS czaring at IPP level
• Set SVN for operational files
• Helped Heather with nfs and eventually started to write a new tool to replace(?) nebdiskd.

Heather Flewelling

• czar (fixing faults, etc)
  • problems with publish - too many faults to allow pub to finish - the faults were all 'old' data that had been cleaned up
• transferred SAS stacks to TD.
• generated plots of times of different stages (and differnt tasks in those stages) for ThreePi.%. This showed that there was a problem in chip stage (slowdown), which Gene found was caused by the mdc parser.
• talked to Jim about John's stacks
• generated plots for # of streaks (by filter and data_group) for ThreePi data. streaks are improved since code changes in April.
• investigated problem in diffs - found bad astrometry not getting caught by psastro. Gene is investigating...
• answered questions to roy about gpc1 database (creating a mock one from scratch). There is some problems generating 'my' minidvodb tables, I need to fix this.

Roy Henderson

• Only worked two days this week (vacation for three).

• IPP to PSPS test loop:
  • set-up ippToPsps (installing Java, Jython, scratch MySQL database etc) on development cluster
  • needed to add some tables and columns to test-gpc1 database (with Heather's help)
  • inserted enough metadata to test-gpc1 about a single camera exposure to run ippToPsps
  • code changes to get IN (initialization) batch working with new Batch class
  • missing primary header values in test smf files. Changed interface code to ignore these and proceed, with errors added to log
  • missing SOURCEID and IMAGEID in .hdr extension: can't interface with DVO without these

• tweaks to czartool after big changes last week
• created PSPS wiki 'News and Updates' page, linked to from PSI
• added some new plug-in queries to PSVO for stacks

Mark Huber

• MD10.V3 work continued, thanks to everyone's help reference stacks have been made, enabled, and distributed to the datastore.
  • reference stacks for yz (higher background) to be available for the first observations of the field at the start of the OC, then gri after Chris implemented the new Dark model. Summary details to be added to wiki page.
  • reprocessing of MD10 exposures through warp of a "better" sub-set of exposures necessary to make reference stacks is available for reprocessing of nightly stacks. The yz filters are complete, gri have ~100 each remain to process.
  • SSdiff testing with new refstacks and automatic choice for convolution direction. Additional testing seems to be necessary.
  • Discussed options for setup of new MD10 for photpipe stack+diffims for transients.
• Further exercise enabling the new RINGS.V3 skycells for 3PI observations and monitoring the nightly processing.
• simtest IPP to PSPS test sample of camera SMFs, stack CMFs, and generated DVO catalogs produced for Roy. Looking into the configuration to recover the missing parts.
• Full Czar experience on Friday with lots of help.

Bill Sweeney

Chris Waters

• Dark: Identified that the previous quadratic temperature model was established for a different (warmer) temperature regime. The current temperature range appears to be in the nearly linear tail of an exponential. This helps explain why fitting with a linear model was yielding less noisy data: a quadratic is generally unjustified by the data. Constructed a new full exposure dark model that uses at most a linear temperature dependence, and confirmed that it introduced less noise than the current dark (~2 counts compared with ~8 counts), as well as being successfully verified against random dark data. Promoted this dark to the currently used model.
• Offnight diffs: These do not appear to be generated at the moment, and it looks like this will require a day or so to fully debug.
• LAP processing: Queued SAS2.123 and SAS.footprint for gri using the new dark. Dropped LAP/14hr y-data to free diskspace and to prepare for a jump to 16hr.
• Diskspace: Rewrote diskspace accounting script to account for diskspace better. Queued chip-stage pattern files for cleanup, as these had not been cleaned before. As they are regenerated by the update process, there is no reason to retain them.
• LAP code: Began rewrite of LAP code to deprecate quickstacks in order to speed up processing.