IPP Progress Report for the week 2010.05.31 - 2010.06.06

(Up to IPP Progress Reports)

Eugene Magnier

One of my current goals is to generate a clean demonstration deep stack of MD04, both for its own sake, and as an example for the rest of the deep stacks. The prerequisite for this is a photometry and astrometry reference database for the field. I a good part of the week working on this, and running into various blocks. I eventually discovered that there were a couple of bugs introduced in the dvo code that corrupted the NAN values, and which in turn resulted in nonsensical magnitudes for some sources. I have addressed this particular issue, and am continuing to double check the relphot analysis on the MD04 field to ensure no other subtle bugs are present in the code.

The telescope finally obtained a set of STS images appropriate for the reference stack, so while I was processing czar on Thursday and Friday, I attempted to build these stacks. The processing proceeded well to the warp stage, but the stacking process bogged down very badly. Our nightly stacks (8-deep) normally only take about 10-15 minutes of processing time. The STS stacks had more inputs (30-60), so I expected them to be a few to nearly 10 times as long. Instead, they took 10-15 hours, much longer than expected, and many failed completely. I looked into this, and consulted with Paul on the possible slow-downs. We discovered three choices in ppStack which are not optimal, and which are exaggerated when the stellar density is high, all related to the fake reference image generated for the PSF matching: 1) there is no limit to the number of stars used to make the fake image; 2) the stellar profiles are placed in the image to a flux significance of 0.01 sigma, which can be extremely large; 3) a separate reference is generated for each input image, even though they cover the same area. Although some care is needed, we can probably make some important speed gains by addressing these issues. I have postponed the STS stacks until at least some of this can be fixed.

Finally, I spent part of the week on a relastro mode to detect possible high-speed proper motion candidates. Niall Deacon has been working on this problem using DVO shell user-level queries, and it has been extremely slow. The relastro version is now working and generates a useful set of output parameters, but is not yet easily configurable (options need to be changed in the C-code). But, where the DVO shell version was taking multiple weeks to run on the 3pi data set, the relastro version can be run in roughly a couple hours for the full sky. It clearly shows that while the DVO shell may be convenient for simple queries, it is far from optimal for queries which are equivalent to a 2point correlation function. PSPS will (hopefully!) be much better in this respect.

Heather Flewelling

  • processing czar tuesday and wednesday
    • set to goto_cleaned ancient cam/fake/warps etc that were never going to process
    • found a warptool bug
    • cleaned up IPP operations wiki
  • finished addtool and merged
    • found a case where dvomerge fails to merge and sent to Gene
  • reported the IPP wiki was down on monday
  • made ISP/GPC1 sky brightness plot for Ken - problems with observations (only had y band GPC1/ISP obsevations), and ISP still looks 'funny', but not like chevrons (not like stars, either).

Bill Giebink

Roy Henderson

  • PSPS
    • Generated more test data for Jim to figure out bug in astrometry code
    • Generated a list of queries to insert frameName for a given frameID for Sue to run
    • Surprisingly large amount of time spent on Wiki configuration (icon, menu, style, permissions etc)
    • Loaded remainder of demo-month 3PI data, thanks to Heather's new DVO database
  • IPP
    • Changed code to take exposure name as an argument and pass it through to the FITS files
    • Numerous changes to incorporate image and detection flags from the IPP
    • Settled a reliable query to pull out all exposures from a given period and coded it into my 'run' script

Paul Price

  • Background restored pipeline:
    • Defined database tables: chipBackgroundRun, chipBackgroundImfile, warpBackgroundRun, warpBackgroundSkyfile
    • Wrote bgtool to implement database operations
  • Changed "all images rejected" error in ppStack to bad data quality
  • MOPS interface format:
    • Reworked ppMops to write SMF-like format (including detection efficiencies), purging duplicates
    • Added functions psFitsReadTableAllColumns and psFitsWriteTableAllColumns to read/write all columns without the additional memory overhead of psFitsReadTable
    • Discovered and fixed bugs in psTree for spherical coordinates
    • Sent example of new interface format to Larry for approval

Bill Sweeney

For Bill, this report covers May 24 - June 5.

  • processing czar 1 day, 1 holiday, 3 vacation days
  • investigated and worked around chip processing problem due to pre-mature schema changes to the gpc1 DB
  • implemented priority ordering for warp stage
  • investigated how a row got deleted from chipProcessedImfile. Failed to find the cause in the mysql binary logs. It is still a mystery.
  • Fixed a couple more cases where the update process stops and the postage stamp server does not handle the error.
  • Did some cleanup of the code in the postage stamp web interface to prepare for further development.
  • wrote a wiki page on shepherding the postage stamp server
  • investigated why 2 postage stamp requests got stuck. It turns out the warpRun's corresponding magicDSRun were in the state failed_cleanup. This prevented the warpSkyfiles from getting queued for update. Cleaned up the magicDSRun faults and the jobs worked. We still need more "revert" tasks to handle cases like these.
  • Fixed a bug that Heather reported where warptool -updaterun didn't work unless the run had been at least partially processed. In the process of investigating this, found and fixed a potential problem with -updaterun -set_state update by disallowing that mode to be used for that purpose.
  • Did some reading on http access control for application to the postage stamp server and data store.
  • Niall reports that he twice tried to submit a postage stamp request and nothing seemed to happen. Investigated this and it turns out his request was over 2MB in size and PHP disallows posts larger than that. Updated the upload page to check for this error (and others) and notified the user.
  • While debugging this I noticed in the apache error logs that there was a poor user who submitted a large postage stamp request on 5/24 and has been constantly querying the data store looking for his results ever since. It turned out that the request faulted parsing and due to a bug in error handling, the request got into a state were I didn't notice the errors. Fixed the problem that caused the error.

Chris Waters

  • Worked with camera team on masking issues. The bad CTE in OTA11 can probably be minimized by changing voltages, as can some of the corner glows that we've been masking in the static mask. Lost columns OTA14 may be somewhat recoverable, as the noise that prompted seems gone at the moment.
  • Added code to nightly processing to queue detrend verification runs from newly taken detrend images. This hasn't been fully tested, and so it disabled in the pantasks task.
  • Finished skycell JPEG code and stack association code. Need to run simtest and debug before merging into the trunk.

Serge Chastel