IPP Progress Report for the week 2010.05.03 - 2010.05.07

(Up to IPP Progress Reports)

Eugene Magnier

I have been focusing on the stack photometry analysis ('psphotStack'). By the beginning of the week, I was happy with the state of the extended aperture photometry, surface brightness measurements, and petrosian values (radius, magnitude, R50, R90). I spent the rest of the week merging the psf-matching code into psphotStack -- the goal is to generate images convolved to a specified PSF and the apply the extended source photometry to the resulting images, using the unconvolved images as the detection image. I managed to get the psf-matching code integrated into psphotStack: the program loads the unconvolved and convolved stack (ie, the stack of the psf-matched input images) and will convolve either of the inputs to the desired target PSF. There is now a bit more work to have the photometry analysis use the correct sets of input images (convolved or unconvolved) for the different analysis steps.

Heather Flewelling

  • out sick for 1.5 days
  • working on addstar merging
  • plots for sky brightness for Ken
  • responding to ifaps1 user requests

Bill Giebink

Roy Henderson

  • PSPS
    • New initialisation batch and detections, complete with photoCodes, for JHU sandbox to test photometry calculations
    • Made and committed changes to the PSPS schema (extra astrometry values)
    • Changed DXLayer code to use exponent form when creating csv files (we were losing precision in astrometry values)
  • IPP
    • Changed 'test' batch to incorporate extra astrometry values needed by PSPS. Made more test data for Jim.
    • Confusion over MSSQL floating-point types (float=double). Updated pspsSchema2XML script, as well as tables/mappings for detections.
    • Adapted my 'czartool' script for ippMonitor, can:
      • show a table of all labels currently loaded in pantasks (stdscience) with numbers for all processing stages
      • show numbers for all states and stages for a particular label
      • show current status of all pantasks servers

Paul Price

  • Noise model fix:
    • Comparison with magictest.%.20100210% reveals fewer streaks, with factors ranging from 0.2 to 1.0 (except at very small counts)
    • Sent before/after fix comparison data to MOPS
    • Majority of streaks in exposure with most streaks appears to be due to bad convolution kernel; might be identifiable using low avg(diffSkyfile.norm)/std(diffSkyfile.norm), but this is unlikely to be generally useful since normalisation is a function of number of input images
    • One effect causing bad convolution kernels is from using (convolved) stacks as templates, where the noise in the stack is sufficiently low as to clip the tops of stars of moderate brightness, and then the diff is forced to use faint stars for stamps
    • Some badly subtracted stars masked CONV.BAD or CONV.POOR contribute to magic streaks; adding these mask bits can reduce streaks, 23 --> 7 in this data.
    • Hough threshold calculation does not include duty cycle. Reducing the Hough threshold by the minimum duty cycle helps find long, faint streaks with low duty cycle. It does introduce more false positives than simply reducing the image threshold to 2.0 (17 streaks from tweak, 9 streaks from threshold reduction), so perhaps a combination of the two would work well? Concerned about impact on shorter exposures.
    • Shorter exposure has 14 streaks from tweaking the code, 19 streaks from reducing image threshold, and 10 streaks without either.
    • Artifacts contributing to streaks in all modes: bathtub rings, diffraction spikes (for bright stars, outer regions aren't masked because angle is slightly off), burn cross-talk.
    • Running all exposures through magic with threshold = 2.0, and separately with code tweak
    • Code tweak is preferred to lowering threshold since the median streak count is 40 vs 50.
    • Sent report to Paul Sydney
  • Extracted and delivered microtest data
  • Jacobian when warping:
    • Average plate scale correction necessary for flux conservation; has been applied for several months
    • Differential plate scale correction shouldn't be necessary, since flat-field correction does that
    • To double-check, processing STS image through warp to check photometry
    • Differential plate scale correction is probably necessary for covariance scale change
    • Chip-warp magnitudes show unexplained offset of ~0.05 mag, and positional variation of ~ 0.015 mag (peak-peak).

Bill Sweeney

  • Wrote and tested code to delete space used by postage stamp requests.
  • Followed recipe provided by Gene to fix the zero points in the ThreePi?.DM.gri dvo catalog.
  • Reviewed my bug list and updated state of tickets. Closed out some obsolete entries, updated some others.
  • Modified streaksremove to set the WCS reference point of the linearized astrometry for released chip stage images to be the middle of the chip.
  • Made various code cleanups in the postage stamp server. Fixed a couple of problems exposed by user requests.
  • Sped up generation of postage stamp request result files.

Chris Waters

  • Finished detectability server. Tested and merged with postage stame server.
  • Enabled automatic processed data cleanup as part of nightly science processing. Data older than the current retention time (30 days for chip and diff, 7 for warp) was manually queued for cleanup.
  • Continued work on revision and mask fraction statistics. Revisions appear to be working correctly, but the mask fraction is cooperating less.