IPP Progress Report for the week 2011.02.07 - 2011.02.11

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• MOPS Processing czar
• IPP czar on Tue and Wed.
• Investigated and fixed bug in ppTranslate.
• Investigating why Nebulous is not efficient. Analysis here ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/neb-touch-gory-details

Heather Flewelling

• lots of magictesting. The numbers look good (mostly lower or the same as the old mask). Outstanding problems:
  • most of the skycells for the stacks I generated have quality > 0, which is bad
  • I still need to look at the magic masks generated for data > 2010-01-24 and data < 2010-07-23.
  • some images have bathtub rings, and I accidentally selected them. The old code doesn't pick up the rings, and the new code does (as magic streaks)
• czar
  • saw some crashes with regtool, which I emailed around.
• ifaps1 work:
  • figured out where the space was used and what I wanted to delete (and emailed around)
• put gene's 3pi dvo db on the rsync server, and answered user's questions
• re-diffed photfest using gene's code, and distributed them
• cleaned up nebulous instances (from processing) with missing files (are these orphans or widows?)
• fixed diftool stackstack queuing bug
• assigned myself some of the tickets, and closed ancient ones that were fixed

Roy Henderson

Bill Sweeney

• Spent most of this week improving postage stamp dependency checking. This part of the system manages the update processing required to regenerate images for the postage stamp server. Like most parts of PS1 this task is made quite difficult due to the requirement that images be run through magic. Deployed the enhanced code late in the week and it is now working relatively smoothly. There are still some bottlenecks that need to be tuned.
• As part of the task above, wrote a wiki page that describes how the state and data_state of IPP processed components are managed. (I needed to review the code to remember how it worked myself.)
• Added code to the IPP tools to respect a special "DO NOT REVERT" fault code, to help try and prevent update processing that cannot complete from continuing to try forever.
• Set up facility to allow IfA postage stamp users to retrieve uncensored stamps.
• Continued to monitor STS processing. Built r band reference stack. Processed the r band images. Johannes reported that some i band exposures were missing. This was likely due to premature cleanup by the operator. Reprocessed them. As of Sunday Feburary 13 all of the STS data from May through October has been processed by the IPP and delivered to the data store.
• fixed some bugs related to muggle postage stamps.

Chris Waters

• Reworked the fringe correction scale determination. There are a large number of outlying points in the distribution of fringe vs. science measurements, which can skew the best fitting line to weird results (the same chip from two MD exposures from the same night had values of 173 and -1.5). The new code fits a line to the bins of points, with the constraint that a bin must have at minimum three points to be included. This results in best fit values of ~10 for both test exposures, and similar plausible values in other tests.
• Nebulous: changed the database to prevent the automounter from attempting to mount disks that no longer exist. This results in a large speed improvement of the nebdiskd script.
• Detection efficiency: Made final change to IPP->OTIS interface that incorporates quality measurement based on the CDF of the det_eff value. Sent CDFs of this value to Ken for both ThreePi and MD. Discovered with Gene that the covariance bug introduces a seeing dependent variation into the detection efficiency values being calculated.
• Registration: Checked that the slowdown at the rawExp advance stage is not a bug, but is due to burntool running slowly. Implemented a --continue option to ipp_apply_burntool_single.pl to allow burntool to be run on sequential exposures in one pass. This should also improve the registration speed. Noted that as the number of dates in the registration pantasks increases, the rate of burntool processing drops due to overhead in checking finished dates.
• Started work on multi-date diffs to attempt to catch pairs that are split between days. Test code finished and runs, but due to weather, there have not been any such split pairs to check.