IPP Progress Report for the week YYYY.MM.DD - YYYY.MM.DD

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• Fixed gpc1 replication. Ingested 2012-03-12T12:05 dump on ipp001 which was seriously broken
• Helped fix nebulous on Tuesday.
• Czar on Wed, Thu, Fri.
• Liaison while Rita and Haydn were at mhpcc.

Heather Flewelling

• dvodb work
  • policy changes for LAP: make separate minidvodbs (cam is one, staticsky is the other)
  • merge staticsky and staticsky multi: now just staticsky.
  • all of this is checked in and tested and recompiled for ippdvo
  • query problem (no solution yet):
    • found query such that if there are 2 cam_ids or sky_ids for a given exp_id (or stack_id), it will take the most recent processing
    • is that what we want to do? Or should we avoid queueing these at all? Should we drop the bad ones so that we don't have the tools 'guess' which is the best (this is bill's idea)
  • sent email to Bill about his ideas for staticsky changes
• sick one day

Roy Henderson

Mark Huber

• Czar Monday and Tuesday
• Caught up with DRAVG minutes
• ppStack work: checked in ppSub:STACK SYS.ERR=0 configuration change to ipp-20120216 once LAP finished and monitored nightly science stacks. Continued looking into the convolved PSF growth in different skycells, filters.

Bill Sweeney

• This was another week working on psphotStack. Progress was finally made on the memory usage problem. The primary issue is that in busy skycells (near the galacitc plane or containing a globular cluster) many large footprints that contain many sources are being created. In the merge process, where the "missing" sources found in one filter but not in another are prepared for forced photometry analysis, these footprints were being copied for each "missing" source. Since some of these were so large - as big as the input images - and complex - hundreds of thousands of spans, and contained tens of thousands of sources, these consumed all memory and swap space. Implemented a cache so only one copy is made for each input and this eliminatedd. However there are certain places in the code which handle these large footprints sub-optimally. For one skycell in the galactic plane it took about 18 hours to complete the analysis. But it did complete.
• Another issue that needs to be resolved is duplicate sources being reported at the position of sources found in pass 1 of the psphot analysis. I found several cases where isolated stars were detected in pass 1, and another weak source was detected in pass 2 which finds detections on an image where the pass 1 sources have been subtracted off. During the psphot matching stage sometimes a perfectly good star is matched with the perfectly good star in other filters, but sometimes one good detection gets matched with the pass 2 source. Thus in the output we will have 2 sources with very different measurements in the different filters.
• Added a feature to the postage stamp extractor program to return an entire copy of the input image. This is useful for two reasons.
  • The LAP stacks are stored with ASIN scaling of the pixel values. The output from ppstamp is linear scaling so it "undoes" that which is useful for examining stack images.
  • The stamp outputs for chip stage replace the inaccurate WCS solution from the telescope, with the astrometric solution from the camera stage.

Chris Waters

• Migrated processing to use the new dark (det_id 853).
• Constructed B-mode dark, and confirmed that it does a better job correcting B-mode data than either the old mixed dark or the new A-mode dark. The slope trends for these three darks are all identical with dateobs, but are shifted relative to each other. No substantial progress made in determining why a given date chooses the particular mode, or why the B-mode disappeared after 2011-05-01.
• Ran script to manually fix corrupt copies related to the ipp064 RAID crash. This should permanently resolve the XY26 issues that we've had in the past.
• Confirmed that UNKNOWN data in the disk usage scripts includes all data not in nebulous.
• Looked at false detections in the dark test footprint using single and 12x detections as the proxies for false and real objects. Applying poor and bad flag cuts removes the majority of "bright" false positives, leaving only a tail that could plausibly be events that randomly move into the 5-sigma detection limit. Comparing these objects to warp and stack images shows that some are in fact real objects, but the majority seem to be streak/burntool/etc variations.