IPP Progress Report for the week 2010.01.04 - 2010.01.08

(Up to IPP Progress Reports)

Eugene Magnier

I have been continuing the study the choice and scaling of the difference image kernels, driven by the level of residual seen in the differences for simulated images with roughly Moffat PSFs. I have implemented a few variations on the standard ISIS kernels to examine this question, and have found that a kernel set that seems to work fairly well: a standard ISIS Gaussians modified by low-order polynomials, supplemented with higher-order radial Hermitians. This kernel set addresses the different fall-offs between Moffats with different scales. For best results, the size range of the kernels needs to be chosen based on the difference of the PSF FWHMs, and it seems to be necessary to avoid integer multiples for the kernel sizes. I have added some statistics to ppSub to track the amplitude of the residuals. I am putting together a write-up of the analysis of the kernels, and will provide a suite of sample images that we can also test against 'Hotpants' or other difference image code.

Heather Flewelling

  • preparing for UKIRT run next week (to observe MD fields)
  • created (in previous week) preliminary scripts for nightly stacks. Tweaked them a bit more, released 12-27, 12-28, 1-04, 1-08.
  • found interesting ripples in z and i band MD01 images. Chris and Gene discovered this was due to Mira.
  • edited recipes: USE STDSCIENCE_VO FOR DATES < 2009-12-09 and DEFAULT FOR DATES > 2009-12-09! Pattern subtraction is on for dates < 2009-12-09, and off for dates after.
  • processing: started SVS.20100108 and SweetSpot.20100108, consisting of SVS and SweetSpot (all) since 12-09-2009.

Bill Giebink

Continuing with DIMM & other hardware work at the summit.

Roy Henderson

  • enabled ability to switch types between IPP and PSPS using mapping file
  • tried to learn enough Perl to write a script to run ippToPsps, without much success
  • ran program through Valgrind, plugged memory leaks
  • generally tidied up, made code safer, comments
  • spoke with Jim and Conrad about example FITS file

Paul Price

  • Catching up following absence
  • Discovered source of ppImage memory bloat: darks are dramatically increasing the noise due to temperature being outside suitable range, causing tonnes of detections in psphot that swamp the memory.
  • Image subtraction
    • Generated hotpants subtraction to compare with ppSub: ~/software/hotpants_v5.1.10/hotpants -inim image.gp.01.ch.fits -tmplim image.gp.02.ch.fits -ini image.gp.01.ch.wt.fits -tni image.gp.02.ch.wt.fits -outim image.gp.01-02.hp.diff.fits -oci image.gp.01-02.hp.conv.fits -oni image.gp.01-02.hp.var.fits -tu 1e5 -iu 1e5 -il -1e5 -tl -1e5 -tr 10 -ir 10 -ko 0 -bgo 0 -ng 3 2 0.85 2 1.70 2 3.4 -r 15 -fi 0 -fin 0 -convvar -savexy image.gp.01-02.hp.stamps -oki image.gp.01-02.hp.kernel -hki -v 2 > & image.gp.01-02.hp.log
    • Verified kernel basis functions convolved with image does not sum to zero, though kernel basis functions sum to zero. This is not due to use of FFT convolutions. We suspect edge and/or numerical effects. Slight reason for concern, perhaps?
    • Penalty = 0 for odd basis functions: encourages basis functions that will produce an artificial astrometric shift. Changed penalty function to sum K2 r4.
    • SYS.ERR and SKY.ERR can really affect the derived normalisation and therefore the quality of the subtraction. Found that turning variance weighting off completely produces a better subtraction (presumably for the same reason Lupton advocates constant weighting for photometry).
    • Artificially forcing single convolution within the framework of dual convolution to verify behaviour.
    • Tracked down error in dual convolution: multiple sign errors.
    • Using SVD with trimming of low-significance singular values, but most important parts to the solution don't have high significance singular values, so tuning required for trim threshold. This is undesirable.
    • Without SVD, matching two Gaussians with different widths results in kernels with a non-zero moment: potentially bad for astrometry (especially of asteroids).

Bill Sweeney

Bill has been working on getting the 'update' (re)-processing to work. This subsystem is required to provide data products to the users after the bulk of the result products have been 'cleaned' (deleted). The basic code has been in place for a while, but it has not been tested; Bill has found a number of bugs and logical issues that he has been sorting out. He has succeeded in getting the 'chip' and 'warp' stage to update, but needs to ensure the updated 'warps' work with the stacking and differencing stages.

Chris Waters

Chris has finished making tweaks to the cosmic ray masking code, but there is an under-lying bug that is now in Gene's court (footprints that are apparently defined, but do not have valid pointers -- these do seem to be real sources, but a mis-interpretation of the flags). He spent part of last week looking into the roll-off effect at the edge of the cells (http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/EdgeRolloff). The main conclusions are that (a) the first/last column(s) of the cells seem to correct well, but the first/last row(s) do not; and (b) this effect has been present since at least run 3, but was previously not a magic trigger because of the 'PATTERN' correction. At most, we need to mask an additional 2-3 rows per cell, but it may not be all cells. Chris still needs to determine what fraction of cells need to be masked.