IPP Progress Report for the week 2009.12.07 - 2009.12.11

(Up to IPP Progress Reports)

Eugene Magnier

I have continued to work on the behavior of psphot in the context of ppSub. I spent part of the week examining the degeneracy of the convolution kernels, and attempting to address the degeneracy with SVD. The plan was to eliminating the modes which did not significantly reduce the chi-square, but this was not very successful: eliminating even the lowest order modes seems to increase the noise in the bright star residuals. My conclusion is that the only avenue for progress in this area is to choose convolution kernels which are more explicitly orthgonal. Since the actual basis function is the kernel convolved with the PSF, this may require an iterative analysis of a range of kernels.

I then worked on the behavior of psphot when operating on the difference images. I fixed a few issues in the visualization that made it difficult to see where psphot was having trouble. That allowed me to identify a specific issue: the source moments analysis, used to guess the psf shape in the positive image analysis, was being re-run on the difference image. Since at this stage, there are few real detections, and most detections are faint, the results of that analysis are suspect. In fact, a typical result is that the sources are mis-classified as possible cosmic rays or defects. The bug that caused this behavior was a failure to pass all of the psf model information to this psphot analysis. By addressing this bug, I was able to demonstrate good photometry analysis of the transient difference sources is simulated images, including consistency with the input magnitudes of those transient sources.

Heather Flewelling

  • re-processed data with dual convolution off. improves # of streaks for STS, makes it a little higher for MD03. This greatly improves max # of streaks for both (from 700 to 400 on md03, from 3800 to 400 on STS)
  • found streakies on magicked images from 20091209 (with camera fixes) - chips 27, 45, 47, 37, 57 dominated. This is data processed with dual convolution, and pattern subtraction turned off. Investigated in more detail with Sidik, Gene and someone else.
  • found IQ measurements for Tonry for a number of exposures.
  • processing various other small labels.
  • fixing user.copies:2 on old raw images < 20080801 before I cull. This is so they can replicate properly.
  • nearly filled up the cluster with processing
  • working on statistics of mask fractions for STS,throughput and warp, when not distracted by other things

Bill Giebink

  • Working on the DIMM hardware for the foreseeable future.

Roy Henderson

  • Continuing work on IPP->PSPS interface
    • Changed code to use psLib for reading FITS tables (rather than the 'FPA' functions).
    • Now have a program that maps smf columns to PSPS columns using an intermediate mapping data file (simple text file for now).
    • Time spent trying to figure out the above mapping, i.e. which IPP columns map to which PSPS columns.
    • Looked into using Perl for XML and md5 for manifest files.
    • Starting to work with Bill's utility functions for extracting info from DVO database.

Paul Price

  • In Edinburgh.

Bill Sweeney

  • added support for 64 bit integer format columns in the Ohana fits I/O code.
  • added code to dvo to include object and detection ids as specified by PSPS to the tables
  • wrote a small utility library for accessing DVO to suport the ipp to psps project
  • wrote a note on the wiki describing how data is identified as it flows through the IPP

Chris Waters

  • Moon glints: compiled plots for telescope meeting on the sky brightness as a function of moon angle. Identified data that shows obvious moon glints.
  • ISP: Unified summit copy and registration for ISP and GPC1, and confirmed that a single summit copy and registration instance will process both cameras' data.
  • psphot CR: Working on code to properly mask out cosmic rays at the psphot stage intelligently. Played with the object classification code while dealing with a bad image.
  • Nebulous: Fixed a bug in nebdiskd that was creating lots of churn in the entries of the mountedvol table. This churn was prompting replication to make unneeded copies of files.
  • Cleanup: Currently pushing 130 labels through cleanup, resulting in a significant increase in available disk space. Discovered issue with diff cleanup where the SQL statement is taking too long to run and times out before completing.
  • Burntool: Identified that burntool is occasionally missing a trail, due either to dropping a burn entirely that has a bad fit, or due to the trail not being fit well. Will investigate more next week.
  • Detrends: Started creation of a new dark for the current camera state after detector changes.