IPP Progress Report for the week 2009.11.09 - 2009.11.13

(Up to IPP Progress Reports)

Eugene Magnier

I have been digging into the astrometry concerns that are causing some problems with both stacking and diff-ing. There are two scales of problems: on one scale, we get catastrophic failures where the resulting astrometry is off by a large amount -- this is caused by the match to the set of reference stars failing badly and choosing the wrong set. On the other scale, we get small-scale failures where most of the reference stars are correctly matched with observed stars, but for one reason or another, the model fit between these systems is poor.

I have made good progress in addressing some of the failure modes for the catastrophic case. This mostly involved fixing the 'clump removal' code to remove all of the significant clumps (these come from regions like the centers of bright galaxies or nebulosity). This fix, along with some tweaks to the number of stars used in this stage, results in a higher rate of successful matches. There is a bit of code already in psastro which compares the measured chip positions with the astrometry model predictions. This should be able to detect any remaining catastrophic failures, and potentially make a better initial guess based on the good chips. This code is not quite right, and still needs some attention.

I also made some progress with the small-scale errors. These seem to be caused by one of two effects (maybe both): (1) having reference stars matched to the wrong source star and (2) lacking enough constraint near the edges for the high-order model. Some of the first case can be eliminated by requiring that a single reference only match to a single (closest) source star. We might also need to limit ourselves to 1st order chip corrections in on-the-fly astrometry calibrations, accepting a bit of lower accuracy on the fly for improved stability of the solution.

I also checked with Paul into the MD08 photometry input images, and was able to prove that the inputs to the stack that Nigel mentioned are NOT contaminated by sky transparency variation. After looking at the psphot psf-aperture magnitudes, I suspect that the psf photometry is being disturbed by having a poor match to the stack psf shape. Paul is going to check that the aperture magnitudes are consistent between stack and warp (and camera).

Finally, we had an incident on the cluster, which has since been resolved: one of the cabinets (containing ipp004,-007, ipp009, a shut down completely. It turned out that both PDU and wall breaker had been tripped. After some investigation, Cindy, Gavin, and Bill decided that the PDU was the culprit and swapped it out. All the machines came back up ok and all the data is fine. However, we have have added a task to Chris' list to define machine groups in nebulous and add the concept that replicas should avoid two copies within a machine group.

Heather Flewelling

  • md08 stacks & diffs (lots of problems with stacks: first all faulted with faults of type 5, then when those were fixed had all faulted with faults of type 4. Paul fixed both).
  • sts finished chips and did stacks
  • started investigations of masks for throughput.20091104
  • fixed burn.tbls that were on 014 (so they would replicate), as well as a few others that didn't manage to replicate.
  • helped bill with realtest (answered questions)

Bill Giebink

  • Continued with realtest work
  • Additional diagnostics on DIMM
  • Recovered/rebuilt office work computer

Paul Price

  • Stacking (branches/pap)
    • Merged into trunk (r26076)
    • Investigating photometry problems
    • Rejection in stacking doesn't seem too excessive
    • Systematic error for rejection stage already set to 0.1 mag!
    • Comparison of stack photometry for the Draper skycell with warp yields 0.1 mag stdev.
    • Comparison of photometry from input warps (even from same night) for the Draper skycell yeilds between 0.06 and 0.08 mag stdev.
    • Gene suspects cause is non-photometric conditions; testing with large MD08 photometry set.
      • Generated addstar environment for operations
      • Added error handling to addstar script
      • Fixed bug in camtool that produced state = 'full' even when fault != 0
      • Getting bugs ironed out for running relphot
    • One of the stars in the Draper stack that Nigel has a variable centroid, by a couple of pixels!
  • Fixed bug preventing stacks working in the wild
  • UK tour paperwork
  • Fixes to stacktool, difftool to support processing

Bill Sweeney

  • Implemented ippTools changes, scripts. and tasks for new cleanup tasks for distribution system.
  • Tested this by running it on some old distribution bundles.
  • Fixed problem with out of order data in the log files
  • Created some new tests for the postage stamp server. Found and fixed a few bugs in the process
  • Started implementing "get_image" mode jobs which will be used to create distribution bundles on demand thorough the postage stamp server. Most of the work is complete. Need to do some code polishing and testing.
  • Spent some time debugging a perl module dependency problem that Joel Welling ran into at Pittsburg.

Chris Waters

  • Vacation for most of the week.
  • Crosstalk: Began work on psastro code to mask crosstalk artifacts. Currently still incomplete and non-functional.