IPP Progress Report for the week 2010.02.15 - 2010.02.21

(Up to IPP Progress Reports)

Eugene Magnier

Heather Flewelling

• automated addstar stage (survey task)
• revert tasks for stack, diff, dist (added to stack.pro, diff.pro, dist.pro)
• investigated magic, destreak, distribution failures- all were nebulous errors, except for a few destreak problems (which Bill investigated - destreak logic problem)
• replicated all detrends (they all have at least user.copies:2)
• liason duties (answering emails)
• answering questions about installing IPP on mac & linux

Bill Giebink

• modified DNSM code to include a "heartbeat" for camera operation
• collected, reduced, and posted DIMM data for the week
• operated RoboDIMM Thursday and Friday night
• performed a couple of CPU transplants on IPP machines

Roy Henderson

• attempts to load data at JHU:
  • prob with objIds - fixed
  • missing SkinnyObject table - added as a new extension
  • permissions/admin problems. Still working on it.
• lots of changes to ippToPsps code after meeting with Gene, Jim etc - all implemented all in ippToPsps
• wrote a perl script to parse PSPS schema in order to minimise work when changes are made

Paul Price

• Propagate zero points from cam to diff. Source lists from ppSub now have calibrated magnitudes.
• Propagate FWHM from warp to diff for ppSub parameter scaling. ppSub now doesn't have to measure the FWHM (at least for warps; not tested on stacks yet).
• Diff failures:
  • MD06.077 failing due to bad reference stack (stack_id = 52606); fixed error code.
  • Every other fault=2 inspected was due to filesystem errors.
  • All fault=5 have dtime_script <= 3 sec, apparently due to difftool unable to find the SQL file.
  • Reworked all ippTools to put fault=2 (SYS_ERROR) when can't find SQL file (presumably due to NFS problems). Removed DATA_ERROR because ippTools don't work on data (astronomical images or catalogues). This should allow us to turn on revert for fault=2 only.
  • Reworked ppSub to be better about error codes and their translation to exit codes.
• Add exposure comment to data store when publishing detections.
• Stack failures:
  • "Stack list contains less than 2 elements" accounts for most of the fault=2. Added "-min_num 4" to automate_stacks.pl and stacktool now requires min_num to be specified and >= 2.
  • fault=5 modes:
    • A bunch of failures in the relative photometry that doesn't make sense on reading the log. They may be due to rebuilding the code mid-way and repeating. Best to revert and retry.
    • dtime_script < ~ 10 are generally due to all (or most) inputs having few lit pixels. Added "-select_good_frac_min 0.05" to automate_stacks.pl to correspond with BADFRAC in the PPSUB recipe.
    • Nebulous errors account for most of the rest, often reading from ipp004 from a range of machines.
  • Reworked ppStack to be better about error codes and their translation to exit codes.
  • fault=4 (PROG_ERROR) seem to be due to failure to find a PSF model, and then triggering a SEGV in ppStack; fixed SEGV.
• Chasing down camera runs in ThreePi.Run5c.20100217 that didn't write all masks
  • 399/2655 = 15% have at least one missing mask (actually, this can be due to bad data quality at chip stage too)
  • Fixed camera_exp.pl to check that masks are written
  • Fixed warp_skycell.pl to ensure mask is available
  • Re-ran camera stage through pcontrol for bad runs

Bill Sweeney

• monitored addstar processing (still a partially manual process) Released 2 versions of 3pi catalog. At the end of the week announced that all catalogs are available on the rsync server
• telecon with Eric Morganson at MPIA, ipp meeting.
• got sick. efficiency reduced for most of the week
• added mode to magicdstool to deal with revert and cleanup faults. tested
• wrote up notes on "optimized update proessing". Spent rest of week working on implementation. It is likely to be easier to finish than originally thought.
• ran simtest to prepare to test changes for update. Found and fixed bug in ppSim.
• debugged and fixed problem with destreak revert
• answered lots of questions about postage stamp server. fixed a couple of bugs.
• started working on optimized bycoord rquests

Chris Waters

• Diffraction spikes: Confirmed that the new diffraction spike code performs better than the old one. There is still some undermasking for the brightest stars, due to the length and added width for these bright sources. Filed a ticket on this for future consideration of adding a variable spike width.
• Stack-stack diffs: Cleaned up overqueued diff runs, and set up the tables used by distribution to handle this mode.
• Difference file names: Added diff_mode to diffRun table, and altered the way the outroot filename is set to allow the type of difference image (warp-warp, warp-stack, stack-stack) to be identified directly.
• Automation: wrote code to handle burntool segfaults more gracefully, but have not yet tested it or committed it to the trunk. Also working on speeding up the stack queueing, so different surveys can be queued up as soon as possible instead of waiting for the slowest one.