IPP Progress Report for the week 2012.02.20 - 2012.02.24

(Up to IPP Progress Reports)

Eugene Magnier

  • added a feature to setphot to pass a set of overall zero point offsets for each filter. This is needed to assign the ubercal solution and then offset the system to the JT recommendation
  • mextract -parallel basically works : needed to add a pass-through to dvo_client for some environment values (TIMEREF, TIMEFORMAT, CAMERA, etc)
  • spent some of the week tracking down bugs in the relphot and mextract processes.
  • started to look more carefully at a test subset of the 3pi dvo db to verify the results of relphot
  • some modifications to dvodist to improve the speed & reliability; started the big dvodist job to spread the full 5TB 3pi dvo db across ~60 machines on the cluster.
  • more work over the weekend to worry about clipping and outlier rejection in the relphot analysis
  • KP12 telecon : got some synthetic images from Peter Draper, looking into comparison of psphot krons with expectations. still getting too small of an aperture...

Serge Chastel

  • Monday holiday
  • Czar on Tuesday and Wednesday
  • Fixed ippdb03 replication crashes
  • Still working on Nebulous checks

Heather Flewelling

  • monday holiday
  • cleaned up JT's list of "Bad 45" - early MD stacks that don't exist: http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/JTMD_graveyard
  • lots of work on collecting MD field information and organizing it in an organized manner: http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/MediumDeepFields
  • investigations of space hogs on ipp005 and ippmanoa. ippmanoa is mostly gene/niall, ipp005 is mostly dvodbs/rocs/nebulous
    • started cleanup of ipp005: this is moving old dvodbs to ippb03.x
    • dvodbs are 'off' until we have disk space on ipp005.
  • processed warps for Larry - need to put them somewhere to grab (they are finished)
  • czar on friday. We were far behind on downloading images (staaaaare). Lots of watching to make sure things were proceeding.
  • showed bill how I make ds9 region files from cmf files
  • in process of repairing the repairable isp files (the files with 1 dead copy and 1 good copy)

Roy Henderson

  • vacation on Friday
  • some big ippToPsps changes to improve usability:
    • configs are now stored in a database table
    • created base-class for all programs which:
      • sets up config and logger
      • handles graceful exiting (including catching Ctrl-C)
      • handles pausing, killing and checking for config changes via Db
    • instead of each loader client queuing stuff up, a 'queue' program writes a list in the db for all clients to see:
      • this required some new tables: pending is stored by 'boxes' on sky so that the new DVO ingest code can be used if necessary
      • these and some other database tables now use foreign key relationships with cascading delete enabled for safety
    • wrote plotter program to create nice density plot of still-to-be-processed stuff for a given config
    • all loaders now automatically choose an available scratch database.
  • PSPS
    • added section detailing Gene's recommended detection cuts to wiki FAQ
    • fixed PSVO plugins to use Sue's workaround for BIGINT detection masks
    • support emails

Mark Huber

  • ppStack cause(s) of overly large PSF in convolved stacks
    • looked at the selected stamps used for convolution
    • moving on to weights and found the SYS.ERR parameter setting may be part of the issue. Tests with order 100 inputs (simulated and MD04) with SYS.ERR reduced back to 0 have a greatly improved convolved PSF size.
    • stumbled across a couple non-critical code bugs
  • looked up some sample regions with brighter stars in MD04 with good and not so good FWHM for Ken.

Bill Sweeney

  • finished reworking the detectability queries to run more efficiently.
  • Started investigating the staticsky failures in earnest. Classified the errors. Most are failing in the "unable to select stamps for psf matching" A couple of assertion failures popped out. We also have some whose memory usage explodes 77 GB is the record so far. That one turned out to have M75 in the skycell.
  • That raised the question "What is happening with the other Messier objects. Found the coordinates and ran a postage stamp request to extract images from the LAP stacks. Did a quick look and for the most part aren't all bad. Further report with images will be written and placed on a wiki soon.
  • Decided that having a tool that would display the skycells for a given coordinate was a good idea. Spent Friday writing this program and used it to examine several NGC galaxies.
  • Started playing with a tool to extract extended source information from staticsky outputs and using it to annotate ds9 display. It is interesting to see where galaxies generate multiple objects and what look like false detections.

Chris Waters

  • Finished work on PATTERN.CONTINUITY, and built a new tag with this in it.
  • Restarted LAP processing at 0 hours.
  • Rerunning SAS2 through the new tag, and distributing it to get comments on false positive rates.
  • Added compute 3 wave to processing host list.
  • Made modifications to nightly_science code to implement offnight diffs and MOPS warp-stack diffs in an automated way. This has not been tested yet, so more debugging work is needed.