IPP Progress Report for the week YYYY.MM.DD - YYYY.MM.DD

(Up to IPP Progress Reports)

Eugene Magnier

  • updated DVO schema to include fields needed for IPP to PSPS: kron mags & errors. may need to update further to support full set of desired mean flux information
  • implemented Peter Veres' / Matt Holman's trail fitting representation in psphot (PM_MODEL_TRAIL). Tested this in simulated images and the fits match the inputs very well. Still need to update the version of the psphot library called by ppSub to trigger this fitting (extended source models) and to handle recipe context for DIFF vs STACK.

Serge Chastel

  • gpc1 replication on ipp001 broken. Dropped database. Started ingestion on Monday morning. Replication back on Tuesday
  • Helped Heather with gnuplot
  • Tuned each mysql server on ipp004-ipp066
  • Installed my nebulous monitoring on ippb00-b03
  • Installed my nebulous monitoring on stsci00-09
  • Modified http://ps1sc.ifa.hawaii.edu/PS1wiki/index.php/PS1SC_Email_Lists

Heather Flewelling

  • czar
    • nothing to note except for lots of trouble on the day I wasn't czar (nebulous).
  • sick 1 day (allergies)
  • ipptopsps
    • rebuilt ippdvo's ipp trunk so it has new addstar/relphot/etc code
    • loaded onto datastore SAS_v7 (after many failures due to nebulous)
    • oh no, a number died at ODM:
      • new addstar/dvo code allows a handful of stack detections to be dropped - these give invalid ippObjID of 0 for stack batches. The 0 ID detections are now dropped and the bad stack batches are being rebuilt.
      • Thomas detected a problem in OBs as well, with about ~100 detections repeated between pairs of OB batches. Heather needs to investigate this but hasn't had time
    • gene fixed up dvo to do kron mags - I got as far as compiling the code but not testing (due to other psps issues and nebulous issues). Intend to test to see that Krons are in there properly, then add them into ipptopsps (for object.magbest)
  • psps
    • estimated number of stacks and stack detections for conrad.
  • dvodb building
    • All ThreePi? smfs are in minidvodb form
    • merging all unmerged minidvodbs into ThreePi?.V3.2012.Q2 - this should be done now but lots of starting and stopping due to nebulous. Anticipate will be done end of weekend.

  • liaison
    • grabbed SAS_v7 smfs for Eddie and put them on ipp001

Mark Huber

  • MD10.refstack generation for starting this OC slowly continues skycell by skycell, working though problems with the tighter input cuts (that also have troubled LAP convolved stacks). Halted on Wednesday with interruption from stsci node mount trouble and deepstack turned off.
  • MD01-10 refstack staticsky photometry calibration with ubercal refcat.20120524 finally finished and available for the next processes for loading into PSPS. Required additional testing and setup of a general configuration for the deeper stacks and fixes to the skycal process. A revised configuration is likely required as well as additional modifications in the skycal process.

Bill Sweeney

  • Continued to work on psphot & psphotStack for stack images.
    • fixed bug where RA and DEC values were not being set in the outputs
    • Changed the code to not fail if one or more of the inputs fails the deconvolution limit test. Since this only affects the convolved stack inputs, changed the code to skip the psf matched radial aperture measurements. This analysis is currently the only place where the convolved images are used in psphotStack.
    • Ran a test with some modified recipe values for a MD09 stack that generates large number of false sources around very bright stars. In the example the false detections are not seen if the recipe value LINEAR_FIT_VARIANCE_MODE is changed from "CONSTANT" to "IMAGE_VAR". Further testing is required to determine if this is a good general solution or not.
    • Verified that the latest version can successfully process skycells in the galactic plane without consuming unwieldy amounts of memory. We still have a performance problem with some inputs. See below
    • The current versions psphot has some sections that take unreasonably long times to process some images in very dense fields. The problem is triggered when very large footprint structures are detected (close to the size of the input image). These structures have very large numbers of spans and some of the algorithms cpu usage blows up. In one example this seems to be triggered when a stack has a small number of inputs so cell boundaries appear in the stacks as NAN pixels. Setting the NAN pixels in the footprint detection image to zero seems to avoid the problem by breaking up the footprints into pieces that can be processed more efficiently. More investigation is required.

Chris Waters

  • Enabled stsciXX storage nodes.
  • Flooded the LAP queue with all remaining runs for this hour slice. Stacks are still being delayed.
  • Defined an ecliptic pantasks and launched a reprocessing of w-band SweetSpot? data for MOPS, to allow 3-year stacks to be generated for differencing.
  • Corrected the high variance print through in stacks by changing the weighting for variances. Instead of using a single image average variance to set the weight for all pixels (as we do when we combine the image data), we now use a simple geometric mean to set the final stack variance. This provides a better estimate, as it prevents the high variance areas from contributing significantly to the final result.
  • Out a day sick
  • Outlining IPP detrending paper.