IPP Progress Report for the week 2010.09.13 - 2010.09.17

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• PSPS: built small test files from existing ones, types conversion concerns

• MOPS: meetings

• IPP: who uses the cluster, helped Roy for mailing-lists.

Heather Flewelling

• ifaps1
  • helped set up Roger
  • answered questions (geoff, dixon, ebeling, tonry, gaidos)
  • found skyprobe data for gaidos
• czar 2 days
  • investigated 131.3PI.g.W+7 chunks with bill
  • tested bills's rundiffskycell.pl - works well
  • changed priority of processing - processing SweetSpots first per ken's advice
  • restarted distribution after it died
• planning trip to Harvard
• attend grad student talks on panstarrs
• addstar
  • merged final ThreePi minidvodb on ipp004. now what?
  • restarted addstar for MD04
• ppMerge -investigating differences between my version of ppMerge and the
  • trunk. They are different, but I'm not sure they are better.

Roy Henderson

• PSPS
  • monitoring of loading. Some failures to look into to (some are the < -30 issue, others need to be looked at)
  • had to stop loading due to issues with DVO Dbs at IPP end

• IPP
  • Czar on Monday:
    • re-queued MDFs from the 11th (were overlooked due to stdscience restart over the weekend)
    • some czartool tweaks (eg log 10 instead of natural log plots)
  • a lot of time spent figuring out mailing list administration. This needs some reorganization.
  • rest of time spent on stacks for PSPS:
    • located example cmf files, worked with Jim to tally IPP tables with PSPS schema
    • some refactoring of my Perl code to prepare for stacks:
      • new class to encapsulate Batch creation/deletion etc. Needed before incorporating stacks
      • batches are now subclasses of Batch, eg DetectionBatch, StackBatch etc. This minimizes code duplication while making code more readable and maintainable.

Bill Sweeney

• Fixed problem reported with postage stamp diff stage image look ups byexp. This was causing problems with detectability queries.
• Debugged problem with destreak -advancerun. Some timing parameters for the task needed to be changed.
• Debugged problem with dvomerge 2mass + ps1 that MPIA was experiencing. It turns out that the 2mass dvo catalog we have distributed (years ago) is incompatible with dvo merge. I also needed to make some changes to the photcodes file that was being used.
• debugged and resolved problem with MPIA's data store where commands behaved differently on some machines. Turned out to be some perl modules missing from the build that were installed on the system on one of the machines.
• debugged some stuck postage stamp jobs. Turned out to be due missing burntool tables and corrupt camera runs (missing mask files) I look forward to the Great Reprocessing so we can avoid these bad data.
• checked in the tools for repairing corrupt image data by rerunning the processing by hand.
• Debugged the fact that some exposures have been added to the ThreePi.V1 dvo database multiple times. It looks like we need to get more clever about error recovery for addstar and dvomerge.
• Two days processing czar.
• Spent quite a bit of time dealing with the special SweetSpott OSS exposures. The desired difference pairs are incompatible with the way we queue diff runs automatically. We also had warp-stack diffs turned on for these data which caused all sorts of chaos.
• investigated a stack failure. Looks like poor quality inputs are not being handled properly in one case. Filed ticket 1427.

Chris Waters