IPP Progress Report for the week 2011.02.07 - 2011.02.11

(Up to IPP Progress Reports)

Eugene Magnier

I spent most of the past week focusing on the reference astrometry & photometry database, making sure relastro and relphot would run correctly on large areas and that they were saving enough Q/A information to enable interpretation of the results easy. I set up a large run of relastro on the big reference database (on ipp045), though it took a couple of tries to get the sizing right. The analysis needs to run on a portion of the sky at a time because there are simply too many measurements to fit in the memory footprint of the single machine. Initially, I was rather simplistically setting the area to be a fix size, ignoring the varying star densities. This was fine for the region near 0,0 (where the script started), but resulted in a memory footprint of 100s of GBs near RA = 270 (the galactic plane). I re-worked the script to use ra-ranges that scaled appropriately based on the number of stars in a region. This is still a bit crude (the program should be smart enough to load a total amount of memory, but that would take more work that I can afford right now; alternatively, it should distribute the load in a balanced way across multiple machines, but that would also be a lot of effort; simple solutions are good enough for now). The relastro analysis successfully ran on the full sky in about 20 hours. Unfortunately, I discovered more recently, that I failed to update an important piece of configuration information in a reference table (the rule for the per-position errors based on the magnitude errors), so I need to re-run that analysis.

I also set up a large run for relphot, using the same concept. I let this run for a few days, before I decided it was going too slowly. It looks like I failed to port one of the recent optimizations from relastro over to relphot -- this particular step wastes 20-30 minutes per pass in assigning images and camera mosaics, and can be done in < 1 minute with the new optimization. I need to fix that and restart the relphot analysis this week.

Given the successful tests of the updated difference code over the past week, I have merged that work from my development branch into the IPP trunk. After a bit more testing this week, we will be able to make a new operations tag.

While the relastro and relphot analyses were running, I have started up again on the field missing from psphotStack output that are expected by PSPS. I have added in the covariances for the extended source fits, and will finish off the rest of the fields. I'll then make a sample MD04 stack analysis for the PSPS team to ingest.

Serge Chastel

Heather Flewelling

  • lots of magictesting. The numbers look good (mostly lower or the same as the old mask). Outstanding problems:
    • most of the skycells for the stacks I generated have quality > 0, which is bad
    • I still need to look at the magic masks generated for data > 2010-01-24 and data < 2010-07-23.
    • some images have bathtub rings, and I accidentally selected them. The old code doesn't pick up the rings, and the new code does (as magic streaks)
  • czar
    • saw some crashes with regtool, which I emailed around.
  • ifaps1 work:
    • figured out where the space was used and what I wanted to delete (and emailed around)
  • put gene's 3pi dvo db on the rsync server, and answered user's questions
  • re-diffed photfest using gene's code, and distributed them
  • cleaned up nebulous instances (from processing) with missing files (are these orphans or widows?)
  • fixed diftool stackstack queuing bug
  • assigned myself some of the tickets, and closed ancient ones that were fixed

Roy Henderson

  • PSPS
    • new method in gpc1 DB class to count number of contributing warps to a given stack
    • discussions with Jim regarding Ken/Gene plan to make a Db with SDSS and IPP stack photometry
    • PSVO
      • Meeting with Jim and Daniel to discuss further PSVO development; some research regarding the plug-in query idea
      • added nice syntax highlighting to the query editing pane
      • fix for issue with large tables from slow queries using up too much memory: now not displaying to screen by default
  • IPP
    • czartool: added ability to choose a day and a survey for the exposure summary page (defaults to 'last night' and 'all' surveys)
    • czartool: improved formatting including highlighting in red when stuff isn't fully distributed
  • Other
    • setup links to metrics from ps1sc wiki
    • finished my IPP report
    • finished my PS1SC blog article
    • some time looking at post-doc candidates' computing experience

Bill Sweeney

  • Spent most of this week improving postage stamp dependency checking. This part of the system manages the update processing required to regenerate images for the postage stamp server. Like most parts of PS1 this task is made quite difficult due to the requirement that images be run through magic. Deployed the enhanced code late in the week and it is now working relatively smoothly. There are still some bottlenecks that need to be tuned.
  • As part of the task above, wrote a wiki page that describes how the state and data_state of IPP processed components are managed. (I needed to review the code to remember how it worked myself.)
  • Added code to the IPP tools to respect a special "DO NOT REVERT" fault code, to help try and prevent update processing that cannot complete from continuing to try forever.
  • Set up facility to allow IfA postage stamp users to retrieve uncensored stamps.
  • Continued to monitor STS processing. Built r band reference stack. Processed the r band images. Johannes reported that some i band exposures were missing. This was likely due to premature cleanup by the operator. Reprocessed them. As of Sunday Feburary 13 all of the STS data from May through October has been processed by the IPP and delivered to the data store.
  • fixed some bugs related to muggle postage stamps.

Chris Waters

  • Reworked the fringe correction scale determination. There are a large number of outlying points in the distribution of fringe vs. science measurements, which can skew the best fitting line to weird results (the same chip from two MD exposures from the same night had values of 173 and -1.5). The new code fits a line to the bins of points, with the constraint that a bin must have at minimum three points to be included. This results in best fit values of ~10 for both test exposures, and similar plausible values in other tests.
  • Nebulous: changed the database to prevent the automounter from attempting to mount disks that no longer exist. This results in a large speed improvement of the nebdiskd script.
  • Detection efficiency: Made final change to IPP->OTIS interface that incorporates quality measurement based on the CDF of the det_eff value. Sent CDFs of this value to Ken for both ThreePi? and MD. Discovered with Gene that the covariance bug introduces a seeing dependent variation into the detection efficiency values being calculated.
  • Registration: Checked that the slowdown at the rawExp advance stage is not a bug, but is due to burntool running slowly. Implemented a --continue option to ipp_apply_burntool_single.pl to allow burntool to be run on sequential exposures in one pass. This should also improve the registration speed. Noted that as the number of dates in the registration pantasks increases, the rate of burntool processing drops due to overhead in checking finished dates.
  • Started work on multi-date diffs to attempt to catch pairs that are split between days. Test code finished and runs, but due to weather, there have not been any such split pairs to check.