IPP Progress Report for the week 2009.11.30 - 2009.12.04

(Up to IPP Progress Reports)

Eugene Magnier

I spent much of the week looking in detail at the interactions between ppSub and psphot. This has led me to explore in detail the behavior of the PSF matching analysis and its impact on the resulting PSF. I have been working with simulated images with known PSFs, and have discovered some issues with the PSF matching code. For example, I was having a lot of trouble getting a pair of images with the same PSF to subtract well. I had not fully appreciated that the PSF matching algorithm is attempting to fit the difference of the two images with zero-flux basis functions (A - norm*B - \sum_i a_i C_i where C_i = k_i x B). Realizing this, I noticed that the raw difference (A - norm * B) had significant residuals -- this meant that the normalization was not measured correctly, partly due to an error in the kernel basis function normalizations (see Paul's report) and partly due to sensitivity of the analysis to outlier pixels. I added a windowing function to limit the normalization calculation to the dominant flux from the matched stars. A related issue is that the coefficients of the basis functions (which are strongly coupled -- this is not an orthogonal basis set) are sensitive to ringing. I have been looking into options for re-conditioning the matrix equation using SVD.

Heather Flewelling

  • processing: when it goes well, I can do other things. This week I spent a lot of time figuring out there was high load on ipp049 (high > 100). Not sure what initially caused the high load, but this caused a number of failures on ipp049 in different stages (diff, destreak, dist), and reverting all at once repeated the high load (and failures). Having as few as 28 hosts and reverting all the diffs can reproduce the high load(!). When there are a lot of faults on the same machine, it is necessary to revert only a few of them at a time in order to get those faults corrected.
  • finished processing MD01 and MD02, processed MD03 and released all three. Processed MD04 and MD05 through to warp, ThreePi?.yz through to camera, and glint for Chris.
  • investigated the bad masks & exposures on (2 of them) for the recent MD03 release. All exposures from this were taken near twilight. One image was the last image taken that night, and the background was really high, and was missing many chips in the camera stage (from various twilight issues). The other image had 2 satellite streaks, and many other 'streaks', but no obvious reason why magic went crazy.
  • sick tuesday

Bill Giebink

  • working on the DIMMs

Roy Henderson

  • Looked at PSPS database schema to see what's required in FITS output from IPP.
  • Looked at 'censorObjects' program as a template for reading in .smf files.
  • Tried to get to grips with autogen/build system of IPP.
  • Wrote a program that can read .smf files and then write out FITS binary tables.
  • Slightly shorter week due to 'New Hire Orientation' and doctor's appointment.

Paul Price

  • On UK tour: visiting Durham (30 Nov -- 3 Dec) and Edinburgh (4 Dec). Freezing.
  • Worked with Nigel Metcalfe and Peter Draper on stack photometry. Discovered (with help from Gene) a bug affecting the normalisation of convolved images. With this fixed (r26320) and Gene's code adding "windows" to the PSF-matching (r26321) on branches/pap, stack photometry is much improved: photometric scatter at the bright end is now smaller than for a warp. Nigel says that the depth scales as expected too. Note that this fix has not been merged into the trunk yet.
  • Discovered that "topcat" is a wonderful tool for working with CMF files: does cross-correlation of tables, plots, subset selection, statistics, etc.
  • Lots of meetings.
  • Currently working on tools to provide PSF instances and coordinate transformations from IPP products.

Bill Sweeney

  • Studied DVO and PSPS database structure in preparation for helping build IPP - PSPS publication system.
  • Implemented code to set a couple of required columns in the dvo db.
  • Debugged and fixed a couple of ipp tools problems.
  • two days vacation to take care of a family member.

Chris Waters

  • Data Quality statistics code: finished writing dqstatstool and associated scripts. This will allow the data quality information determined by the IPP to be returned to OTIS to aid scheduling.
  • Nebulous: installed updated nebulous code in ippdb00 and ippdb02. Looked into outstanding bug for deleting objects with no available instances.
  • Sky brightness/moon glints: Looked at the correlation with observed sky brightness with angular separation from the moon. Checked to see how moon glints influence magic streak detection.