IPP Progress Report for the week 2010.06.21 - 2010.06.25

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

Heather Flewelling

• ippMonitor work
  • detrend verification histograms for darks are up on the test ippMonitor
  • more bug fixes
• detrend server - sheparding detrend verifications (darks)
• dvomerge - we moved the master to ipp037 when ipp004 ran out of disk space. merging minidvodbs from ipp004 to ipp037 was too slow, so Gene and Paul cleaned up space, then I got sick, and I moved back the master to ipp004 to restart the merge on monday (merge is restarted now).
• gave Serge a grand tour of IfA and MIC during the cluster power outage on Wed.
• out sick 2 days

Bill Giebink

Roy Henderson

• PSPS
  • mysteriously wrong ra/dec for certain chips and exposures: generated test data for Jim to work with
  • changed schema: renamed moments in various tables and views, removed momentTheta from ImageMeta

• IPP
  • changes to configuration after schema change (above)
  • now storing image flags from DVO in ImageMeta
  • wrote script to poll ODM web-service and query batch status for everything loaded, then update local Db with results. This is the basis for the recovery system.
  • Processing Czar for two days:
    • faults shown in Czartool now link to the correct log files
    • added 'last night's data' page in Czartool: quick summary of data taken at summit vs that registered at MCPCC
    • updated Processing Czar wiki page with recent changes

• Other
  • lost time to power shutdown on Wednesday

Paul Price

• MOPS support:
  • Discovered several problems with diffphot output which will require a re-run
  • Fixed psModules to read/write PSF clump parameters
  • Fixed psphot to produce meaningful anti-dipole stats when loading a PSF
  • Fixed psphot to read WCS, so CMF file includes celestial coordinates
  • CR_NSIGMA, EXT_NSIGMA not being set for sources because of lack of clump parameters in input PSF --- can't do anything about it, and not a big deal anyway
  • Fixed psphot to set rough classification of 'STAR' where the clump parameters don't exist
  • Fixed psphot to respect mask when doing linear fit, otherwise photometry not done for sources lying in area with advisory mask
  • Re-running diffphot (6226 diffs)
  • Fixed ippTools to respect magicked status properly
  • Publishing (magicked) diffphot results to IPP-MOPS-UPDATE on DataStore
• New production build: ipp-20100623
• Background restored stages:
  • Added scripts and tasks
  • Debugging/verification run on simtest
  • Merged into trunk
  • Adding destreak/distribution
• Cleaned out data on ipp004
• Investigating processing failures
  • Re-ran several runs with corrupted files (this is annoying!)
  • Updated FITS table reads to deal with empty tables for removestreaks
  • Set runs to 'drop' where inputs have been cleaned

Bill Sweeney

Chris Waters

• IPP-OTIS interface:
  • Determined realistic FWHM and extinction criteria from CDFs of these values from science data taken since Demo Month
  • Added automation handler to queue new bundles every hour.
  • Created bundles for past data from before the automation take over.
• Cleanup. Changed the cleanup handling to be less monolithic, allowing us to clean different surveys and stages at different rates. Added distribution cleanup with a 7-day retention time.
• Began work on OTA14 masking status. Generated a large number of images to check to see if we can recover columns 0,2,3.
• Edits to nebulous:
  • Provide a note to be set for each volume, so we have a record of their status.
  • Implement neb-migrate, to remove data from a specified volume.
  • Modifications to neb-ls to make it more user friendly.