IPP Progress Report for the week 2011.04.18 - 2011.04.22

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• MOPS czar
• IPP czar
• Update of IPP-MOPS ICD: ICD Lite wikipage now contains the ICD pdf and nothing else...
• Documenting and fixing hemocrat
• MOPS detection efficiency
• bzip instead of gzip for MySQL dump archives of gpc1, ippadmin, and ippRequestServer. E-mails are now sent to ps-ipp-ops@…

Heather Flewelling

• started rsync of ThreePi db to ipp003 - it stalled out at some point because ipp003 got rebooted, and I restarted it. Not sure if it is complete yet.
• addstar -resorted, relphot'd, and merged the 3 MD04 databases for Roy. (/data/ipp005.0/gpc1/catdirs/MD04.merges/MD04.merge)
• fixed a couple things on ippMonitor (add*)
• czar monday
• sick wed and thursday
• state holiday friday

Roy Henderson

Lots of progress, and a major setback, getting stacks into PSPS this week. Still lots to do before Boston.

• Finished remaining development work for stack batches:
  • questions about how to populate PSPS flux columns: Jim and Gene settled on a solution and I implemented it in code
  • numerous schema changes
  • work on GPC1 queries to list available stacks, get contributing OTAs for each
  • new temp Db table to store meta data from DVO, eg flags, photcode etc
  • redesigned ippToPsps database to keep track of batch types, processing, whether they are loaded or merged etc
  • fixed issue of NaNs in fits headers not writing to database
  • changes to lots of SQL due to Heather's changes in gpc1 addRun table
  • now performing clean-up in Db before export to FITS, i.e. removing NULL objIDs, NULL fluxes etc

• Successfully tested loading and merging of new stack batches. Problems encountered:
  • missing tables: SkinnyObject, ObjectCalColor in stack batches. Added these to code.
  • NULLs sneaking in to new 'updated' column in Object table: we had forgotten to add a default. Changed schema
  • crazy -999 "NULL"s don't fit in byte field and so broke load. Fixed
  • merge stage was expecting all contributing images for the stack to be loaded already. Sue fixed this.

• DVO speed issue
  • encountered a serious DVO speed issue with the new MD04 database. Way too slow for us to load enough before Boston
  • formed plan to pull everything we need into a MySQL database for quicker, easier access
  • started work coding this

• the solution to the above DVO problem means my new code needs to produce detections, so:
  • lots of work filling in the last missing fields
  • porting GPC1 access methods in Jython code
  • laboriously comparing the output with that of the old code to verify nothing has been lost along the way

Bill Sweeney

• Bill spent most of this week finding a workaround to the bright star astrometry problem for the STS exposures. This involved:
  • studying the addstar implementation getting familiar with how it works.
  • changing the code to (optionally) use different sets of stars for the initial grid search and final fitting
  • Tested using the usual range for grid search and for the fit using stars within a magnitude range where the errors in the astrometry are small.
  • Ran tests of the results in dvo. Created a wiki page showing the results for our test exposures.
  • tested that new code is not use with the usual recipes and integrated into the production tag
  • Started reprocessing the STS.2010 data using the new code.
• investigated problem with queueing staticsky runs.
• One day and part of the weekend as processing czar.
• Investigated and fixed several problems with the cluster that unfortunately occurred over the weekend.
• searched for and queued data for cleanup.

Chris Waters

• Reprocessing/large area processing
  • Confirmed that even when we have a smaller number of inputs, pairwise diffs still returns better magic results than diffing against a quickstack.
  • Began design work on large area processing tool and scripts. This should manage all the processing for exposures with minimal manual intervention.
• Astrometry: Processed second microtest data. Details available here: ​http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/GPC1_BrightStarAstrometry_SW_LH_scan
• Started work getting Nebulous to see new large storage hosts at the ATRC.