IPP Progress Report for the week 2012.02.13 - 2012.02.17

(Up to IPP Progress Reports)

Eugene Magnier

  • created a test suite for parallelized relphot & setphot
  • tests of relphot & setphot in parallel mode + code fixes : both now get the right answers
  • spent some time over the weekend putting together a first pass for a parallel version of mextract

Serge Chastel

  • ipp003 nebulous import finished (successfully)!
  • Nebulous with Heather.
  • Found incorrect files on

Heather Flewelling

  • dvodb
    • answered dvodb questions
      • bertrand
      • roger
    • the usual: making sure LAP/ThreePi are continuing. Tricky because of lots of stops
    • restarted rsync of minis to ipp0222 - copied the newest ones first for bertrand
    • isp dvodb - restarted because it was using the wrong camera in the scripts (does this matter?)
      • suspect something is off/goofy: It takes > 12 hours for some of the dbs
    • ipp005 is getting too full (21T gpc1, 6T neb, 11T roc are the top offenders)
    • ipp022 is getting too full (have not investigated the offenders)
  • isp
    • continuing to process it (mainly for addstar purposes)
    • transferred files to ifaps1 for rjw
  • JTMD stacks
    • investigated about 1/3 of the "Bad 45"
    • MD01/MD02 finished, MD03/MD04 in progress
  • nebulous investigations
    • 12% of isp files are affected
    • fixed half of those (of those half, less than 1% are dead/unfixable)
    • worked with Serge on other nebulous investigations
  • czar
    • found a detrend in the same funky state as isp files (good copy/bad copy)

Roy Henderson

  • ippToPsps:
    • testing of new ippToPsps loading mechanism
    • found bug when ingesting DVO files at 0/360 boundary: fixed
    • made mechanism for purging batches from system:
      • added a column to database called 'purged'
      • changed to cleanup progran to delete 'purged' batches from datastore and local disk
    • fixed bug in DVO ingest class occurring after a previous failed ingest attempt
    • added client info to ippToPsps Db:
      • each client now writes pid, host, config to database
      • can pause or kill a client from Db
    • all programs now catching Ctrl-C for graceful exiting
    • started loading 3PI (aka LAP) using new code. No problems so far.
  • Documentation:
    • wrote a detailed wiki page of notes about loading for PSPS ORR
    • added a PSVO section to FAQ and a link to it from the main PSVO help page
  • Support:
    • a lot of email support for Peder and Nigel in Durham, mostly re PSVO
    • some help for Kent Wood
    • more mailing list stuff (Roberto's list of names for pspswg)

Mark Huber

  • ppSub testing:
    • with revised and more comprehensive test set, unfortunately found that revised convolution direction choice is actually more erratic than correct.
    • local photpipe setup for preparing IPP images for and comparing output with HOTPANTS. Processing of the revised MD04 test set completed.
  • ppStack write-up and wrapping up initial simulated data tests.
  • A little kicking of LAP.
  • Minor update to TSS page.

Bill Sweeney

  • Spent this week rewriting the scripts that execute detectability queries for the postage stamp server. The previous version ran all of the jobs in the same job. For actual user requests this can take several hours and so if anything goes wrong, the job must be re-run from the beginning. Modified to system to create a job for each individual input component. These can run in parallel. Almost finished with this restructuring. Testing is in progress.
  • Finished processing and distributing the STS PP5 data set with greatly improved astrometry. MPG is in the processing of evaluating the results.
  • Two days as processing czar.

Chris Waters

  • Background: Ran SAS data through new code, and distributed the SAS three year g data. Looked at M31 data to confirm that the continuity code helps in that situation. This pointed out that a small number of bad cells can skew the entire OTA's data. This was handled by adding a check that refuses to correct a cell if it is sufficiently different than its neighbors. This seems to resolve the issue.
  • False positives: checked that in single exposure data, we do see a decrease in the number of false detections with the new continuity code.
  • Other work: Found SMF files for JT, ran SS WS diffs for MOPS.
  • Set up new tag that includes continuity correction, although it's on hold for LAP to finish (which is now done) and for a bug fix in the continuity code (which is not).