IPP Progress Report for the week YYYY.MM.DD - YYYY.MM.DD

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• IPP
  • ippdb02 tests
  • Bug in ipp_serial_inject.pl. The uri of files to be inserted is wrong: ​file://path/to/data (instead of ​file:///path/to/data)
  • Fixed files monitoring
• IPP/MOPS
  • Set up the ipp framework on mops14 for synthetics analysis. All necessary modifications to the original IPP are in ipp@mops14:~/dev/ipp_modifications. I also have detailed notes on the installation process (all the files and database operations which are required to have a running ipp): now I just need to write them down in a wiki. On Friday I had been able to run the full processing (till mops diff products). Two concerns though: (1) there is still a step which is necessary to be manually run between chip and cam stages; (2) Processing a quad gave me 3 products instead of 2. I have to investigate why.
  • I haven't been able to install scipy with Python 2.6.5 which is by default on the system so I installed 2.7 for the ipp account.
  • Started to set up the ipp for megacam images processing. I created a dedicated account for that: ippmc@mops14. I successfully got registration running (thanks to Heather and her 1504 ticket). When I tried to run MegaCam images through the IPP though, it stopped at chip stage with weird half empty logs. I'm stuck at this point now.
  • I also created a dedicated account that will be used for Subaru image processing: ippsu@mops14
• MOPS
  • I'm thinking of using what I did about 2 years ago, that is, Condor for the IPP to replace pantasks. Two reasons for that: (1) I will not have to modify the high-level IPP scripts (e.g. chip_imfile.pl) which are necessary to process the synthetics. (2) Modifications to the regular processing will likely be easier (through the use of DAGs) if we want to add or perform some stages differently (e.g. Steve Hartung's diff).

Heather Flewelling

Mark Huber

• data gofer for Hartung visit
• MD_WSdiffim_photcheck_test20121004

Bill Sweeney

• Spent most of this week putting the finishing touches on the software for tracking ipp releases. One output of this will be a web site that may be used to access camera stage detection files (smfs). The difference between this and the standard distribution system is that it does not require second copies of the data to be stored on the cluster and thus can be used "permanantly".
• Corresponded with Eddie Schlafly, one of the primary consumers of the new smf retrieval system to make sure that it will meet the needs of the ubercal project.
• In preparation for populating the release management tables investigated the frequency of duplicate LAP runs in the current release. Wrote scripts to hide the extra results.
• Designed a method to manage the completion of lap processing for a region. This includes a new lightweight table called lapGroup. This will be used
  • to signal that a set of 5 lap runs for a given projection cell have completed
  • that it is time to queue the staticsky runs
  • and that staticskyRuns have been queued for those skycells.
• Worked a bit on the ipp side of the new PSI postage stamp web service.
• two days as processing czar
• consulted with Chris as to how to fix problem where lap was blocked because cleanup was not running.

Chris Waters