IPP Progress Report for the week 2011-07-04 - 2011-07-10

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• IPP czaring explained to Mark
• MOPS Czar (Fri -> Fri)
• IPP Czar (Friday -> Tuesday, Friday)
• MD IPP-MOPS-TEST publishing issues
• SVN server for IPP operation (to be continued)
• Hsieh mops smf
• ippdb02 crash (not recovered yet)
• Huntley support for stares
• Mailed IPP-users concerning gpc1_0/gpc1_1 future

Heather Flewelling

• czar for 3 days
  • many problems - registration/burntool/summitcopy problems on wed
  • nfs/nebulous/many machines down on thursday
  • stopped processing twice (once for dump, once for memory upgrade)
  • cleaned up lots of blood on czartool (see czar log for details)
  • mark - gave him instructions on processing MD10, some czar hints as well.
• isp
  • after discussions with ken - possible way to tag the exposures that have been obscurred by the dome - this requires more testing, I was interuppted by czarring
• dvodbs
  • ThreePi - merged ThreePi.124 - ThreePi.340
  • rsyncing over this db to ipp005
  • LAP - queued all available destreaked files
• holiday monday

Roy Henderson

• ippToPsps loading:
  • some minor troubleshooting, eg missing IQ_FW1, FW2 values in psf header causing problems for moments in ImageMeta
  • some dxlayer issues:
    • lost config used for Boston loading, csv path for loader was wrong
    • daemon quitting after a single batch. Had to keep restarting
    • Conrad to the rescue: tested and working now
  • issues finding the correct stack cmfs to load:
    • apparently we make two cmfs per stack_id. Changed SQL accordingly
    • impossible to find the full path for a given stack from GPC1, so wrote code to read all cmf headers at a given path_base until correct one is located
  • issue with skycell ID: we need a schema change, as we store as an int, but format is eg 0683.043. Mangled to 0683043 for now
  • loaded all stacks, not that there were many

• ippToPsps coding:
  • started work on some loading metrics code including:
    • queries to find numbers for pending batches, processed batches, faults etc
    • queries to monitor speed of loading, time of last published
  • sub-classed python Logger class in order to add some convenience methods, like printing separators, value pairs
  • improved a lot of the logging messages for consistency
  • code changes, as well as changes to ipptopsps Db, mean different batch types are handled polymorphically. Much cleaner
  • added a warning prompt if 'force' and 'publish' options are both enabled
  • encapsulated FITS reading code into a class. Constructor now copies file to the local file-system first to speed-up access
  • started work on jython code to query ODM via SOAP for batch status.

Mark Huber

Bill Sweeney

Chris Waters