IPP Progress Report for the week 2010.08.09 - 2010.08.13

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

* IPP:

  • Working on SQL optimizations in ippTools: I created a local copy of gpc1 on wednesday. The idea is in the end to have a small database which might be used for tests (likely not performance tests). I started to look at the various queries nut nothing fancy until now.
  • Tested the trac sectionedit feature and helped Cindy installing it


  • Tested pyfits (FIPS files) and suds (SOAP Python layer to request similarly to PSI): both seem to fulfill what I expect. I'm currently building the Python framework for the unit tests. The purpose is to check if IPP2PSPS inputs are the same as the outputs obtained by querying the database.

* (Passport: I should go to San Francisco on 9/7 (I'm still waiting for an answer from French Embassy though))

Heather Flewelling

  • MOPS
  • ifps1 liason
    • transferred massive numbers of files for Zagursky
    • is it possible to make a direct connection between Maui <-> ifaps1?
    • answered many questions
  • ppMerge debugging
    • learned how to use gdb
    • generated many different types of test cases for ppMerge
  • started on ppSub simulations

Roy Henderson


  • successfully restarted loading to Maui cluster:
    • minor bug at PSPS end prevented loading, Sue fixed it quickly
    • a change in IPP proxy server prevented access to datastore
    • some minor DXLayer issues
    • at this point, MD 3,4,5 and 6 and a lot of 3PI have been loaded
  • Helped Jim investigate mysterious object ID assignment in DVO
    • created mosaicked images for relevant chips, found offending detections
    • meeting with Gene and Jim to discuss


  • czartool
    • big changes to incorporate labels from update server
    • fixed fonts on czartool plots (can now see host names)
  • ippToPsps
    • added a Datastore class to perl code
    • DVO access very slow. Did some tests and discovered network IO was main issue, eg 9 min on ipp016 vs 2 min on machine with DVO Db
    • some work on my ODM polling script, now using ippToPsps Db class methods for query and update
  • Czar on Friday:
    • reprocessing of MD01 warps and nightly stacks with new tessellation, with help from Heather and Chris
    • MD01 changes to night science config and stdscience input file

Bill Sweeney

  • Fixed various problems in the update process revealed by a storm of 250000 postage stamp requests submitted by the MOPS team.
  • Two stressful days as processing czar.
  • fixed some bugs in distribution of background restored data.
  • started working some scripts that users may use to create postage stamp request files and detectability query files from DVO.
  • tracked down slow data transfer speeds to the consortium to an overloaded http proxy.
  • tracked down cause of "403 forbidden" data store errors to flaky ipp cluster machines going down.
  • helped two visitors learn to create and submit postage stamp requests.

Chris Waters

  • Linearity: reprocessed all data to correct bug in previous runs (dark was applied), and fixed software correction. This system seems to work now (http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/DetectorLinearity). Will merge with IPP trunk next.
  • Automation: Added mechanism to create SweetSpot? stacks to automation code. Updated stack-queuing mechanism to correctly notice irredeemably failed camera runs and queue stacks from the remaining data. Shortened the retention time for Three Pi chip and diff data to 14 days, to allow a large free space buffer.
  • Detectability: Fixed bugs that were preventing data from being updated. Have not fully tested the changes. Updated wiki page (http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/detectability_ICD_proposal) to reflect syntax of a more relaxed version 2 query.