== IPP Progress Report for the week 2011.04.25 - 2011.04.29 == 
[[PageOutline]]

(Up to [wiki:IPP_Progress_Reports IPP Progress Reports])

=== Eugene Magnier ===

=== Serge Chastel ===

 * MOPS czar
 * Replication: all databases on ippdb01 are replicated on ippc02 and on ipp001.
 * Slow log query supervision now works (see SlowLogSupervision database on ipp004). I need to make it a daemon to update the tables contents and display them in czartool.
 
=== Heather Flewelling === 

 * addstar
   * addstar was moved to new ippdvo account
   * restarted minidvodbs
   * copied old ThreePi.V1 to /data/ipp006.0/gpc1/catdirs (others live here
   too), and deleted the original off of ipp004 to free up space
 * JTMD stacks
   * MD02 and MD03
   * found and fixed bug in ippconfig for old data
   * found and fixed bug in detRun and dettools - detselect does not always choose the correct detrend when the use_date is null
 * MD04 stuff
   * answered questions on addRun/staticskyRun
   * added in fake addRuns for Gene's MD04 deepstack dvodb
 * isp astrometry
   * doesn't work - same runs as before, but things have changed in the recipes
 * sdss
   * downloading ra/dec/mags/etc for comparision to 3pi data
 * czar for 2 days
=== Roy Henderson ===



 * DVO -> Mysql program written and working:
  * loops through all DVO sub-dirs, pulls in all measurement and average tables, joins and inserts into master dvoDetection table
  * problems with duplicates in DVO - ignoring for now, will investigate later
  * various system problems: broke ipp005 by building big database in the root partition, moved to ippc00 (too slow) then back to ipp005 after Gavin moved Dbs to partition with some space
  * completed ingest of 167 million detection rows in about 30 hours. Query speed increased from 40 minutes for a full frame to around 20 seconds.

 * Testing of detection loading using new MySQL database:
  * path problems - csv dir not understood by ODM
  * DXLayer defaulting to 3PI rather than SCR
  * confusion over IMAGEID key: IMAGEID in smf files is not same as IMAGE_ID in DVO. Using SQL workaround.
  * crazy -999 "NULL" overflowing byte fields and breaking load
  * !ImageMeta extension existing for empty Detection extension: fixed
  * duplicate detections in smf files breaking load: altered my tables to reject them and report to the log
  * nightly stacks in DVO do not have header values that were in original stack demo file. Changed code to query gpc1 for them instead
  * deep stacks not in addRun table (they were loaded manually) so I couldn't queue. Heather to the rescue...
  * found rows in !StackApFlx and !StackModelFit not in !StackDetection: ignoring for now

 * Actual loading of detections:
  * required lots of hand-holding over the weekend, but necessary to get everything loaded in time.
  * encountered bug causing program to halt with a 'database user' prompt. Spent a lot of time diagnosing issue, then fixing it (STILTS functions not closing all their database connections)
  * investigated high numbers of bad flux, duplicate objIDs. Confirmed same result with old code.

 * made plot of magic mask fraction across lunations for Ken

=== Bill Sweeney ===
 * Spent many hours tracking down a memory corruption problem in addstar when importing SDSS data. It was very difficult to find since the problem only happened in an optimized build. Code was wiping out a saved variable on the stack.
 * Fixed problem with postage stamp web form where user errors were not being reported in a useful way.
 * Investigated the cause of some missing keywords in static sky cmf files that are needed for PSPS. Tracked the problem down to ppStack. Fixed it.
 * Analyzed results of tests of new STS fields. Did some tests of yet another STS tessellation. This one with 7 projection centers.
 * Did some debugging of some postage stamp requests that got stuck waiting for updates.
 * One day as processing czar.

=== Chris Waters ===