IPP Progress Report for the week 2011.08.15 - 2011.08.19

(Up to IPP Progress Reports)

Eugene Magnier

  • continued work on extended source analysis : finalizing fitting mods and kron apertures for large galaxies
  • data rsyncing for second wave of disk installs & finishing first wave
  • more investigation of processing throughput problems

Serge Chastel

  • 3-day week (Thursday off)
  • Cleaned IPP-MOPS-TEST and IPP-MOPS with entries older than 2011-07-01
  • Rewrote cleandsproduct which uses a date instead of an publishing entry (~ipp/local/bin/cleandsproduct.py)

Heather Flewelling

  • liason stuff
    • md stacks for TD, masks question
    • wiki information for MH
    • ipp instructions for WBest
  • dvodb stuff
    • start LAP.ThreePi?.20110809
    • rsync ThreePi? over to ipp006
    • rsync all ipp004 catdirs over to ipp006
    • restart ThreePi? dvodb on ipp006
  • czar for 2 days
    • nothing special to note except that we did chip.off to let the nightlyscience finish
    • unsticking some exposures for Mops

Roy Henderson

  • czartool:
    • moved code out of trunk/tools and into ippMonitor; made relevant changes to czartool webpage; now running as ipp from ~ipp/src/ippMonitor (i.e. not from my home dir anymore...)
    • added path field for plots in config so that czarpoll can save to any NFS accessible location, i.e. not /tmp on ipp004
    • changes to nebulous plot to show all ATRC machines
    • stopped generation of plots that are not displayed anymore - sped things up a bi(t
    • added two new pantasks servers: 'stack' and 'addstarlap'
    • updated documentation accordingly
  • ippToPsps coding:
    • now making plots of loading per DVO database. Helps to visualize progress
    • lots of new static methods for batch classes to ensure batch names and paths are consistent across all programs
    • wrote cleanup program. Checks for batches merged but not deleted then removes them from local disk, datastore and DXLayer
    • now writing total detections to database and logging initial and final totals of detections for each chip, as well as totals for the whole exposure
  • loading:
    • investigated issue with IN batch reported by Thomas: binary column being populated with a string: Fixed
    • speed issues: crazy-slow for galactic center. optimized some SQL updates, but likelihoods are the bottleneck. Needs work
    • some time investigating duplicate object IDs across chips reported by Thomas
    • began, and completed, loading of available LAP data
  • other:
    • discussion and emails with Kent Wood (NRL)
    • investigated issue of corrupted VOTable files from DRL reported by Dan Wood (NRL): Fixed

Mark Huber

  • In reviewing the MD10 stacks:
    • noticed that they were made as compressed versions and will remake uncompressed versions once another issue is resolved.
    • investigating some skycells with extreme rejects prompted from Gene a suggested modification to the match rejection in ppStack. Working through testing it.
    • attempted quick, unconvolved deep stacks but needs more work.
  • MD01 refstack creation and reprocessed sample of warps, nightly stack continues.
  • Still testing ppSub choice for convolution direction.

Bill Sweeney

  • created vptool the ippTool to manage the new video photometry stage. Wrote script to ippScript to launch the program psvideophot for all chips for an exposure that have a video cell.
  • two days as processing czar.
  • fixed problem with publishing where runs were queued that cannot be processed.
  • debugged problems with new M31 reference stacks

Chris Waters

  • LAP: Bug fix to correct issues with old exposures not having a valid chipRun queued. Began work on automatic run queuing, to allow for the system to manage the number of active runs itself (which will eliminate the need to manually add runs and should speed up the effective processing rate). Attempting to sort out other speed related issues to determine why LAP isn't processing as quickly as needed.
  • ROC: Finished debugging final repair script. This appears to detect nearly all types of ROC file failures, and repairs/reconstructs/replaces entries as needed. Still need to run it on a large chunk of the ROC data.
  • Diskspace: launched cleanup for old distribution data that we do not need sitting around still.