== IPP Progress Report for the week 2011.04.18 - 2011.04.22 == 
[[PageOutline]]

(Up to [wiki:IPP_Progress_Reports IPP Progress Reports])

=== Eugene Magnier ===

=== Serge Chastel ===
 * MOPS czar
 * IPP czar
 * Update of IPP-MOPS ICD: ICD Lite wikipage now contains the ICD pdf and nothing else... 
 * Documenting and fixing hemocrat
 * MOPS detection efficiency
 * bzip instead of gzip for MySQL dump archives of gpc1, ippadmin, and ippRequestServer. E-mails are now sent to ps-ipp-ops@ifa.hawaii.edu
=== Heather Flewelling === 

 * started rsync of ThreePi db to ipp003 - it stalled out at some point because ipp003 got rebooted, and I restarted it. Not sure if it is complete yet.
 * addstar -resorted, relphot'd, and merged the 3 MD04 databases for Roy. (/data/ipp005.0/gpc1/catdirs/MD04.merges/MD04.merge)
 * fixed a couple things on ippMonitor (add*)
 * czar monday
 * sick wed and thursday
 * state holiday friday
=== Roy Henderson ===


Lots of progress, and a major setback, getting stacks into PSPS this week. Still lots to do before Boston.

 * Finished remaining development work for stack batches:
  * questions about how to populate PSPS flux columns: Jim and Gene settled on a solution and I implemented it in code
  * numerous schema changes
  * work on GPC1 queries to list available stacks, get contributing OTAs for each
  * new temp Db table to store meta data from DVO, eg flags, photcode etc
  * redesigned ippToPsps database to keep track of batch types, processing, whether they are loaded or merged etc
  * fixed issue of !NaNs in fits headers not writing to database
  * changes to lots of SQL due to Heather's changes in gpc1 addRun table
  * now performing clean-up in Db before export to FITS, i.e. removing NULL objIDs, NULL fluxes etc

 * Successfully tested loading and merging of new stack batches. Problems encountered:
  * missing tables: !SkinnyObject, !ObjectCalColor in stack batches. Added these to code.
  * NULLs sneaking in to new 'updated' column in Object table: we had forgotten to add a default. Changed schema
  * crazy -999 "NULL"s don't fit in byte field and so broke load. Fixed
  * merge stage was expecting all contributing images for the stack to be loaded already. Sue fixed this.

 * DVO speed issue
  * encountered a serious DVO speed issue with the new MD04 database. Way too slow for us to load enough before Boston
  * formed plan to pull everything we need into a MySQL database for quicker, easier access
  * started work coding this

* the solution to the above DVO problem means my new code needs to produce detections, so:
 * lots of work filling in the last missing fields
 * porting GPC1 access methods in Jython code
 * laboriously comparing the output with that of the old code to verify nothing has been lost along the way
=== Bill Sweeney ===
 * Bill spent most of this week finding a workaround to the bright star astrometry problem for the STS exposures. This involved: 
   * studying the addstar implementation getting familiar with how it works.
   * changing the code to (optionally) use different sets of stars for the initial grid search and final fitting
   * Tested using the usual range for grid search and for the fit using stars within a magnitude range where the errors in the astrometry are small. 
   * Ran tests of the results in dvo. Created a wiki page showing the results for our test exposures.
   * tested that new code is not use with the usual recipes and integrated into the production tag
   * Started reprocessing the STS.2010 data using the new code.
 * investigated problem with queueing staticsky runs. Proposed a solution, but got no feedback yet.
 * One day and part of the weekend as processing czar.
 * Investigated and fixed several problems with the cluster that unfortunately occurred over the weekend.
 * searched for and queued data for cleanup.

=== Chris Waters ===

 * Reprocessing/large area processing
  * Confirmed that even when we have a smaller number of inputs, pairwise diffs still returns better magic results than diffing against a quickstack.
  * Began design work on large area processing tool and scripts.  This should manage all the processing for exposures with minimal manual intervention.
 * Astrometry: Processed second microtest data. Details available here: http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/GPC1_BrightStarAstrometry_SW_LH_scan
 * Started work getting Nebulous to see new large storage hosts at the ATRC.