== IPP Progress Report for the week 2009.09.13 - 2009.09.18 == 
[[PageOutline]]

(Up to [wiki:IPP_Progress_Reports IPP Progress Reports])

=== Eugene Magnier ===

=== Heather Flewelling === 

 * Processed MD09.20090910 through magic, sent email to harvard with link to data
 * Processed M31.20090917 through magic.
 * Processed MD10.20090915 through chip stage, half fail at cam stage (investigating, some psastro, some ppstats related. ppstats related to burntool state)
 * Processed M31.2008.20090918 through chip stage, half fail at cam stage (investigating...)
 * burntooled all dates between 06-01-09 and 09-16-09
 * edited [wiki:PS1_DataRequestManagement wiki] to keep track of which data I'm processing: 
 * checked md5sum on 568794 gpc1 images, found 6 corrupted images, removed them and replicated the good copy.

=== Bill Giebink ===

 * Continued to identify and correct errors in psLib unit tests. Basic procedure is to:
  * identify failing tests,
  * determine if the test is still a valid measure of the underlying code, and,
  * if the test is not measuring anything useful then it's eliminated or replaced, also,
  * failures due to changed function parameters, different error checking, etc. are fixed.

 * In addtion, commented upon test deficiencies are being remedied where feasible.

=== Paul Price ===

 * Detection efficiency: merged branches/pap into trunk, r25383.
 * PS1SC:
  * Set up nightlogs@ps1sc.org list
  * Organising UK tour
  * Retrieved and sent dravg archive
 * Stacking:
  * Everything was getting rejected.  Turned off "additional variance" in pmStack, and get decent rejections now.  This is not well understood yet, but may be connected with a change made a month ago removing the "variance factor" terms, or renormalising the image subtraction chi^2.
  * Can identify bad images using the iq statistics calculated in psphotImageQuality.c: iq_m2, iq_m2_uq-iq_m2_lq, iq_m3 and iq_m4 are useful quantities.
  * Sent Nigel and Peter a dump of ThreePi data from June/July that should be useful for determining good cuts on these quantities.
  * High-frequency structure in kernels caused by minimum ISIS width = 1 (FWHM, not sigma!): fixed ppSub recipe
  * Increased rejection threshold (COMBINE.REJ) to 2.5 from 2.0 standard deviations.
  * Using a given analytic PSF instead of that produced by pmPSFEnvelope produces reasonable results for the stack.  Attempting to correlate resultant convolution kernels with iq statistics.
 * Fixed fuzz of FITS images so that expectation values are preserved (trunk@25439), after help from http://fits.gsfc.nasa.gov/registry/tilecompression.html



=== Bill Sweeney ===

 * Split the database queries for magic de-streak processing. The old way was taking too long and thus causing jobs to run multiple times.
 * Modified summit copy and registration scripts/tools/tables to save the md5sum and size of the compressed raw images. This will allow us to detect potential data corruption problems.
 * spent some time analyzing the results of the MD09 streak detection.
 * added new states to destreak processing to automate restoring destreaked images and to censor destreaked files that are found to contain unmasked streaks
 * restructured the queries for distribution bundling. The old way was taking too long and thus causing multiple jobs to run for a given component
 * worked with Johannes at MPG to set up a data store for submitting postage stamp requests at that site.
 * Fixed some bugs in postage stamp extraction triggered by requests submitted by Queen's University.
 * Built distribution bundles for the reprocessed and new MD09 exposures. Posted them to the ipp datastore.

=== Chris Waters ===

 * Began pushing old data through the chip and warp cleanup process.

 * Fixed resulting bugs in the cleanup code that did not arise during
  simtest tests.

 * Added burntool_state column to rawImfile table, and rewrote
  ipp_apply_burntool.pl and chip_imfile.pl to handle the new burntool
  versioning system.

 * Still working on getting register_imfile.pl and ppStats to
  successfully identify if the summit has run burntool, and if so,
  set burntool_state correctly in the database.

 * Identified new bug in burntool when run in a two-step process.