IPP Progress Report for the week 2009.09.13 - 2009.09.18

(Up to IPP Progress Reports)

Eugene Magnier

Heather Flewelling

• Processed MD09.20090910 through magic, sent email to harvard with link to data
• Processed M31.20090917 through magic.
• Processed MD10.20090915 through chip stage, half fail at cam stage (investigating, some psastro, some ppstats related. ppstats related to burntool state)
• Processed M31.2008.20090918 through chip stage, half fail at cam stage (investigating...)
• burntooled all dates between 06-01-09 and 09-16-09
• edited wiki to keep track of which data I'm processing:
• checked md5sum on 568794 gpc1 images, found 6 corrupted images, removed them and replicated the good copy.

Bill Giebink

• Continued to identify and correct errors in psLib unit tests. Basic procedure is to:
  • identify failing tests,
  • determine if the test is still a valid measure of the underlying code, and,
  • if the test is not measuring anything useful then it's eliminated or replaced, also,
  • failures due to changed function parameters, different error checking, etc. are fixed.

• In addtion, commented upon test deficiencies are being remedied where feasible.

Paul Price

• Detection efficiency: merged branches/pap into trunk, r25383.
• PS1SC:
  • Set up nightlogs@… list
  • Organising UK tour
  • Retrieved and sent dravg archive
• Stacking:
  • Everything was getting rejected. Turned off "additional variance" in pmStack, and get decent rejections now. This is not well understood yet, but may be connected with a change made a month ago removing the "variance factor" terms, or renormalising the image subtraction chi².
  • Can identify bad images using the iq statistics calculated in psphotImageQuality.c: iq_m2, iq_m2_uq-iq_m2_lq, iq_m3 and iq_m4 are useful quantities.
  • Sent Nigel and Peter a dump of ThreePi data from June/July that should be useful for determining good cuts on these quantities.
  • High-frequency structure in kernels caused by minimum ISIS width = 1 (FWHM, not sigma!): fixed ppSub recipe
  • Increased rejection threshold (COMBINE.REJ) to 2.5 from 2.0 standard deviations.
  • Using a given analytic PSF instead of that produced by pmPSFEnvelope produces reasonable results for the stack. Attempting to correlate resultant convolution kernels with iq statistics.
• Fixed fuzz of FITS images so that expectation values are preserved (trunk@25439), after help from ​http://fits.gsfc.nasa.gov/registry/tilecompression.html

Bill Sweeney

• Split the database queries for magic de-streak processing. The old way was taking too long and thus causing jobs to run multiple times.
• Modified summit copy and registration scripts/tools/tables to save the md5sum and size of the compressed raw images. This will allow us to detect potential data corruption problems.
• spent some time analyzing the results of the MD09 streak detection.
• added new states to destreak processing to automate restoring destreaked images and to censor destreaked files that are found to contain unmasked streaks
• restructured the queries for distribution bundling. The old way was taking too long and thus causing multiple jobs to run for a given component
• worked with Johannes at MPG to set up a data store for submitting postage stamp requests at that site.
• Fixed some bugs in postage stamp extraction triggered by requests submitted by Queen's University.
• Built distribution bundles for the reprocessed and new MD09 exposures. Posted them to the ipp datastore.

Chris Waters

• Began pushing old data through the chip and warp cleanup process.
• Fixed resulting bugs in the cleanup code that did not arise during simtest tests.
• Added burntool_state column to rawImfile table, and rewrote ipp_apply_burntool.pl and chip_imfile.pl to handle the new burntool versioning system.
• Still working on getting register_imfile.pl and ppStats to successfully identify if the summit has run burntool, and if so, set burntool_state correctly in the database.
• Identified new bug in burntool when run in a two-step process.