IPP Progress Report for the week 2010.11.01 - 2010.11.05

(Up to IPP Progress Reports)

Eugene Magnier

I investigated and solved the problem we were having in generating dark models which we believed. Earlier in the summer, when we attempted to update the dark model to better follow the temperature variations, and to use the "new" camera temperature keywords, we found that the residuals produced by the analysis were substantially worse than expected: lots of visible structures in the jpegs and average residual flux which was not close to zero. Disturbingly, the mean of the residuals was not roughly zero, which seemed to suggest something was going quite wrong with either the model or the application. I examined the various steps of the process, and discovered that the code was working as we expected and intended, but it was the reported residual statistics and jpegs which were in error. It turns out that the code which was measuring the statistics was also creating an internal image mask, but because of the way that mask was generated, negative pixel values were considered 'BAD' and masked. As a result, even though the residual images were quite clean and had ~zero mean flux, the statistics (and jpegs) measured for these images were always biased high. This particular effect only shows up in this case because other stages of the processing either have non-zero bias (raw images) or have an input mask from an earlier stage. Fixing this problem, I was able to generate darks (using inputs from nearly the full semester) which leave very nice looking residuals.

I also fixed some code issues in the non-linearity correction code. There was a segfault due to ancient code triggered by turning on the choice to correct non-linearity. The code also ran much too slowly, so I re-optimized it to make a single scan of the correction table. The non-linearity correction now can be used, but some further tuning of the correction measurement is needed to generate a reliable correction for the cell sag.

Finally, I have started to tackle the repair of the big ThreePi? DVO database which was corrupted when the relevant disk filled up. This revealed placed in the dvomerge code which were not correctly catching and handing the data I/O failure. As for the database, I have created to tool to examine the existing images and measurements, and to identify images for which measurements were dropped. These images and the remaining measurements can then be removed from the database to make the tables consistent again. The dropped images can then be re-added to the database and dvomerge can continue as before.

Serge Chastel

  • Updated czartool page with replication status
  • Added wiki page about how replication problems were solved (link from the czartool page)
  • Contributed to to Heather's IPP wiki documentation
  • Investigated XFS defragmentation concerns
  • Tried to figure out how OSS data are published and to see where they are hiding between warp and diff stages
  • Working on psLib 'make test' and trying to make it work with Hudson

Heather Flewelling

Roy Henderson

  • PSPS
    • loading:
      • stopped 3PI, and loaded remainder of MD04
      • troubles starting 3PI loading again with primary DVO Db: corrupted. Spent some time investigating this
      • loading is now stopped pending a new 3PI DVO Db
    • investigated current state of DVO MD04 Db with Heather. Verdict: needs more exposures
    • added extra filtering to ODM-checking script: can check those only on the datastore, 'MergeWorthy', 'Merged' etc
    • communications with TOPCAT developer about a plan for the PSPS Toolbox

  • IPP
    • Czar on Monday and Tuesday
      • the strange case of the missing OSS exposures
    • IPP Metrics in czartool:
      • new StageMetrics class to encapsulate metrics for a stage for a given time/label
      • new method in Gpc1Db class to get a reliable count of summit exposures for a given date
      • new method in CzarDb to get a full list of labels in use for a given time period
      • script to use above classes to generate HTML page of metrics for a given day, with plots
      • new stacked histogram showing rate of processing over time for each stage

Bill Sweeney

  • Queued data for MD04 reference stack.
  • Monitored processing of MD04 data. Large numbers of warps and stacks triggered bugs that had to be worked around.
  • pestered the processing czars with suggestions.
  • dealt with various cluster problems
  • Debugged problem with magic processing. Somehow an invalid magicRun got entered in the database. Didn't find root cause.
  • finished implementing background run cleanup. (This is more time consuming than it would seem. Lots of ippTools changes).
  • Debugged MPIA data store problems.
  • investigated some postage stamp server problems.
  • one day as official processing czar.
  • defined new tessellations for STS and M31.

Chris Waters

  • Nebulous: limited the shuffle code that is moving second instances to large free disks to prevent causing excessive load on either the destination computers or the nebulous database server. Started work on a neb-fsck that will be used to ensure that the disk and the database agree. Miscellaneous bugfixes to neb-admin to allow shuffle and replication to run simultaneously.
  • IPP-OTIS interface: added a bit of code to start sending more information than a simple binary acceptance field. Most likely result will be to send the CDF value of the current exposure's measurements so OTIS can decide at what level to retake an exposure.
  • automation: unlinked the MD stacks and OSS diffs from each other, allowing each survey to proceed after checking that its own prerequisites are met. This seems to work correctly, and should get OSS diffs out a few hours earlier. Fixed some typos that may have been preventing the magic/stack blocking from running (which means that stacks are made from destreaked data). Will have to check later to see if this resolved the issue.
  • burntool: moved permanently missing storage objects out of the way, and reran burntool for these dates. This should resolve the issues with update.
  • diskspace: ran diskspace analysis code, noting that mdc config files are now #3 user of space. Started running compression on old darks flagged by Gene.
  • linearity: code works faster after Gene's refactoring. Need to confirm that it applies the scientifically correct values.