IPP Progress Report for the week 2011.01.10 - 2011.01.14

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

Heather Flewelling

• magic test - running diffs/magic on r28950 for Gene, see ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/magictest201101 The numbers are similar (a little better) than 20100701 tag.
• wrote a script to trace an exp_name through all the stages of processing. It is located in tools/heather/expbot.pl
  • using this script I found a number of problems. ThreePis NOT being processed, ThreePis and Quads not being properly diffed. See ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/ProcessingInvestigations2011 for more information. Chris is debugging the ThreePi bug.
• talked to JT about databases and such for his SN cluster
• answered Matt(s) questions (harvard liason)
• investigated fringe some more, see ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/FringeInvestigations2011 and ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/FringeInvestigations2011.part2. It's not obvious why some otas explode, and it's not dependent on cell.
• addstar: We stopped merging some time ago because relphot was crashing on a minidvodb. relphot doesn't crash on later minidvodbs, so I have requeued up the faulty minidvodb for addstar processing. I will see if the new one also causes segfaults. More information can be found here ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/ThreePiMinidvodb.283
• I lost my keys, and Raja found them.

Roy Henderson

• PSPS
  • discussions about, and small amount of research into, the design of a new DRL
  • loading continued, with some issues: batches with duplicate objIDs and dodgy flux values discovered by loader
  • PSVO
    • new classes, with common base-class, for slow and fast queries so that they can be handled polymorphically, meaning much cleaner code
    • PSVO can now receive messages via SAMP. Whole tables and table selections, including graphical selections on sky-plots in TOPCAT, can be passed back to PSVO for further query to PSPS
    • now remembers username/password combo to avoid logging-in every-time
    • began documentation page on PSPS system wiki
• IPP
  • Czar wed/thurs. Minor issues
  • czartool
    • substantial time spent introducing a long-overdue XML format configuration file across the czartool code
    • brought documentation up-to-date

Bill Sweeney

Chris Waters

• Back from vacation, caught up on status.
• Fixed bugs in new nightly science code:
  • race condition identified in burntool processing due to images that exist at the summit but are not downloaded returning a NULL instead of a valid class_id. Using the summitImfile.class_id value fixes this issue, ensuring proper ordering of images.
  • updated regtool -pendingburntoolimfile calls to use a book of dates instead of just calculating the current UTC date. This solves the issue of images from one day being ignored if they did not complete registration correctly by 14:00 HST.
  • updated the register_imfile.pl script to not bother burntooling images taken during the day to speed up their registration. This is controlled by two variables in the register.pro task: $sunset = 03:30 and $sunrise = 17:30. These should be generous enough to not cause problems.
  • identified bug that was likely preventing stacks and diffs from being generated (the script was not correctly checking if more processing was needed at end of night, and was claiming it was finished queuing stack and diff tasks). Fixed this, and determined that some diffs were being done out of order due to a quirk of the SQL dropping other incompletely processed data. Fixed this as well.
• Looked at neb-cull, and determined that there was no bug in the code. neb-cull requires a volume, not a host name, which explains why the ROC script that was ran previously did not correctly cull the appropriate instance. Wrote correction script for the ROC files that moves the bad instances to the appropriate volume, and then corrects the ROC file to point at this new instance. Added roc -validate mode to read a ROC file, and check each input file and calculate if there are any data errors.
• Fixed log(flux) scaling to generate images with the expected scaling.