IPP Progress Report for the week 2010.01.11 - 2010.01.15

(Up to IPP Progress Reports)

Eugene Magnier

We have been working this week on two main fronts: 1) finishing some development work to be ready for the 'demo month', and 2) producing nightly stacks for the Medium Deep fields.

I spent much of the week working with Paul to make sure the kernel matching process is as good as we can reasonably make it, especially the dual convolution portion. Paul put together a nice suite of examples on Friday that compare the now current ppSub parameters with our best understanding of Hotpants (ppSub_vs_Hotpants). In that example, ppSub does quite well. However, we would not be surprised if the difference imaging experts in the consortium can provide better Hotpants parameters. The big question is if they can demonstrate better performance than the ppSub performance.

I have also been working on a psphot bug that Chris W identified -- his cosmic ray masking code was running into some segfaults when operating on the footprints for certain objects. I tracked this down to a premature free of some of the first-pass footprints. This fix is still on the eam_branches/20091201.

I have also started to build new detrend masters based on the new camera settings. Two things have become clear in this work so far: 1) sometimes the camera does not report a GAIN value for a cell, and we have been treating these cells as bad -- that may not be the case, so we are going to modify the analysis to treat those cells as having a GAIN of 1.0. 2) It is clear that we will need to update our static mask -- there are changes in the hot column / glowing defects with the new settings, and some clearly need to be masked, though some old masks may now not be needed.

Heather Flewelling

  • answered questions harvard asked me (before I left for the big island)
  • handed off nightlystacks scripts to chris
  • started header scripts for John Tonry

Bill Giebink

Roy Henderson

  • IPP/PSPS interface
    • Memory problem - psFits/Lib functions using huge amount of memory. Now writing tables per CDD (as in spec) instead of one big detections table.
    • Got Maui cluster access, learned how to locate diffs/stacks etc.
    • Made changes to FITS file structure based on meeting with Conrad.
    • Speed issues - psFits/Lib very slow. Four minutes to write 75Mb file. Optimised some code, but more needed.
    • Now writing out FITS for diff stage (many fields missing, to be decided upon).

Paul Price

  • Fixed bug in generation of skycell and mosaicked camera configurations that was preventing warp update.
  • Tracked down bug in publishing to MOPS: sources with NAN magnitudes are not being propagated, but most sources in the CMF files have NAN magnitudes, and those that don't appear to be spatially correlated (!?). Gene suspects this issue has been solved by a recent fix to psphot.
  • Image subtraction with dual convolution
    • Seems to be a problem in math for the unconstrained least-squares problem: reworking the least-squares matrix in a simpler manner yields a solution that does much better!
    • Fixed sign error in penalty function.
    • Created more complete test set
    • ISIS_RADIAL, PENALTY=1, SKY.ERR=SYS.ERR=0, DUAL=TRUE with orders (1.8,2)(3.3,2)(5.8,2)(10.2,2) very successful in producing zero sources in output photometry except for large mis-aligned input PSFs.
    • Using widths = (pi/2)i for i = 1..6 and order = 2 reduces the problems with large input PSFs, but has some nasty over-subtractions.
    • Comparing with hotpants. ppSub with dual convolution looks better (as one would hope: more free parameters)
    • ppSub about twice the running time of hotpants, but doing 4x the convolutions (2 input images, 2 variance maps)
    • Wrote up on wiki: http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/ppSub_vs_Hotpants
  • Fixed bug in update of diffs: wasn't reading in the old kernel, so was calculating a new one. ppSub on trunk now produces identical output when doing an 'update'.

Bill Sweeney

  • Finished testing and integrated changes to the IPP to correctly mange update processing and destreaking
  • Gathered up detrends and created a set of destreaked burntool applied raw images for the M31 team at MPE.
  • tested injecting the detrend and raw images into an ipp database outside mhpcc. Discovered a problem with destreaking of raw images. It tured out to be a tricky bug recently introduced in the streaksremove.
  • Travelled to Germany for a 2 week visit to MPE and MPIA.

Chris Waters

  • Rolloff analysis: http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/EdgeRolloff Generated statistics to look at which chip/cell combinations have worse rolloff issues. Still need to finish this analysis.
  • Nightly stacks: Wrote code to automatically create nightly stacks. It identifies the completion of summit_copy, applies burntool to the data if needed, queues the chip->warp processing, and queues the stack processing when the warps are complete. More testing would be useful. Current processing listed at: http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/NightlyStacks
  • Moved nightly stack data off of ipp026 in case hardware problems cause it to lose data this weekend.