== IPP Progress Report for the week 2010.06.14 - 2010.06.18 == [[PageOutline]] (Up to [wiki:IPP_Progress_Reports IPP Progress Reports]) === Eugene Magnier === === Serge Chastel === === Heather Flewelling === === Bill Giebink === === Roy Henderson === * Build issues with !FitsConverter on Maui cluster. Usual suspect: cfitsio. Fixed makefile. (PSPS) * Changes to the way I publish batches to the datastore to match up with Conrad's new DXLayer (IPP) * a fair bit of work to correct the exposure times in the PSPS Db: * hacked a version of my ippToPsps code to run through all currently loaded exposures and pull out exposure times then write SQL updates to scripts (IPP) * checked these SQL scripts into svn for access from SSMS (PSPS) * ran each script (one for !FrameMeta, one for Detection) on each 'cold' Db on each of the six servers (very slow and frustrating) (PSPS) * Off sick for three days === Paul Price === * Have psAbort() and psAssert() log a backtrace before bailing: attempting to track down thread bug * MOPS support: * Moved current IPP-MOPS !DataStore to IPP-MOPS-20100614 * Reverted ppMops in branch ipp-20100610 to produce 'new old' format (with columns for anti-dipole stats, but not SMF-like) * Turned off automated cleanup to keep as many diffs as possible for re-photometry * Added more search options to pubtool -definerun * Republishing all !SweetSpot and 3pi diffs since 2010-02-01: pubtool -definerun -rerun -client_id 1 -data_group !ThreePi.% -data_group !SweetSpot.% -dateobs_begin 2010-02-01 -dbname gpc1 * Added new analysis task: diffphot * Added feature to ppArith: binary operation with a constant instead of an image * Fixed pubtool definerun and pending queries so they can find cleaned diffs * Re-republishing all !SweetSpot and 3pi diffs since 2010-02-01 (now including the cleaned ones): pubtool -definerun -rerun -client_id 1 -data_group !ThreePi.% -data_group !SweetSpot.% -dateobs_begin 2010-02-01 -set_label mops.20100616 -dbname gpc1 * Queued 5293 exposures (~30k skycells) for diffphot: diffphottool -definerun -set_workdir neb://@HOST@.0/gpc1/mops.20100617/ -set_label mops.20100617 -set_note 'Re-processing diff photometry for MOPS' -data_group !ThreePi.% -data_group !SweetSpot.% -dateobs_begin 2010-02-01 -dbname gpc1 * Approx 10% of diffphots fail (fault=4) because diff PSF doesn't include clump parameters * Realised diffphot doesn't respect magic * Preparing to publish results * Fixed small coordinate bug in pmPSFEnvelope * Excessively large values in database dumps * MySQL bug: values of DBL_MAX in database are dumped with insufficient precision and rounded outside of legal bounds * Added code to psLib to catch these and truncate to FLT_MAX * Identified 52 instances in database; Bill fixed * Fixed dsproductls: MOPS' !DataStore doesn't include the comment header, so removed overzealous checking that content is non-empty * Discovered and fixed (I hope) race condition in threading system * Fixed psMessageDestination to append log/trace messages (finally!) === Bill Sweeney === === Chris Waters === * Finished, committed, merged, and did final debugging on the warp/stack/diff summary tools, and on the stack association code. * Nebulous work * Wrote code to migrate instances off of a specified volume * Wrote API to find which instances are on a given volume * Looked into the distribution of slow database queries, with the goal of shifting the timeout from 1s to 2s (although 3 or 4 wouldn't be excessive). * Processing czar, with a focus on queuing old data for cleanup.