IPP Progress Report for the week 2010.05.10 - 2010.05.14

(Up to IPP Progress Reports)

Eugene Magnier

I spent most of the week finishing off the basic stackphot analysis code. I finished the PSF-matching code for the stack photometry, and updated the extended source analysis to work within that context. I also added circular apertures for all sources, regardless of extendedness. I added a new output CMF format for the stack photometry, PS1_SV1, which includes those radial apertures. I have generated a test data set for one of the MD04 fields and released that to the community (see Stackphot_20100514).

Heather Flewelling

  • addstar minidvodb progress
    • finished writing addtool -arguments, these work the way I want them to, I just need to write the scripts and tasks for them
  • processing czar on Thursday and Friday
    • updated check_system.sh to check update pantasks - so we don't forget it when we stop/restart things
    • Bill, Gavin and I had tons of trouble sorting out problems relating to home dir move (current working directories like /data/ippc18.0/home/heather would strip down to home/heather, this was eventually sorted out with a reboot of ippc18)
    • rebuilt the ipp -clean -rebuild -optimize (after the home dir move)

Bill Giebink

Roy Henderson

  • PSPS
    • Long meeting with Conrad regarding the design of the recovery system
    • Successfully loaded new data on sandbox, and then Maui. This was a test of various changes from last week including:
      • new 'init' batch and detections with photoCodeIDs
      • double-precision astrometry
      • exponent-form CSVs
      • schema change with new astrometry values
  • IPP
    • Processing Czar for two days
    • Added extra stuff to Processing Czar wiki page
    • czartool on ippMonitor:
      • Can view the status of any pantasks server
      • Can show whether each stdscience label is currently being distributed or not
      • Fixed image viewing stuff that I broke when installing czartool

Paul Price

  • Processing czar:
    • o5320g0023o (MD04): each source surrounded by pupil; looks like it's supposed to be a focus image, and there were hardware problems. Set chip 91857 to 'drop'. obs_mode is 'MD' but comment is 'qtfocus' and filter is 'OPEN'. Complained to ps-obs, ps-otis.
    • Fixed locking problem in pmSourceFitSet.c
    • Rebuilt production build; restarted pantaskses
    • cam_id = 48386,48401 subject to catalogue pole bug; set to state = 'wait'
    • Re-ran chip 87061 XY66 because file corrupt
    • stack_id = 80154,80002,60001,80103,80143 have non-existent inputs; set to state = 'drop'
    • Set stackRun.state = 'drop' for 243 stacks with fault=5 (generally "insufficient inputs", etc).
    • Set stackRun.state = 'drop' for 38 stacks with fault=2 where the inputs were unavailable
    • Fixed stacktool -tosum to properly select stacks that have all inputs available
    • Fixed sporadic error in psImageConvolve.c where thread task is undefined when called by psphot
    • Set up /local/ipp/ on ipp037
  • Noise model fix:
    • Ran exp 125537 through magic to check for faint streak detection
    • Streak is detected by all 3 magic options
    • Ran magic v2.4 (2010-05-12) on verification suite for final approval
  • Chip vs warp photometry:
    • Running exposure with Jacobian applied in pswarp
    • Offset is now -0.02 mag (was*0.05 mag), but the variation of the difference with position has been greatly reduced.
    • Outliers are concentrated in some areas.
    • Checked fix to pswarp into branches/pap
    • Report sent to ps-ipp-users
  • Working on PSF-matching paper

Bill Sweeney

  • For Bill, this was a week to clean up various odds and ends.
  • Added option to dsgetfileset to decompress files when downloaded. Requested by a postage stamp user.
  • Update get_image postage stamp jobs to work with the restructured postage stamp system system. Users may now request bundles of destreaked data through the postage stamp interface.
  • Discovered a problem due to changes in the way diff outputs are structured that caused updates to fail. Fixed it by retrieving the original location from the database and putting the files there during updates. This required changing the tasks and scripts at each stage of the pipeline so spent a bit of time testing this thoroughly using simtest.
  • Tracked down problem caused simtest to generate too many stars. Checked in a workaround and filed a ticket for the root cause. (config file problem).
  • investigated problem where dvoImagesAtCoords reports that a given RA and DEC is contained in multiple chips on an exposure. The root cause is due to using a transformation outside of it's valid range. Consulted with Gene and he suggested a fix that will take some time to implement. Since the problem is rare, filed a ticket and defered the problem for now.
  • Spent several hours with Heather and Gavin investigating a strange problem where processes on the cluster lose track of their current directory. It may be due to the home directory relocation confusing nfs.
  • Fixed a couple of bugs triggered by users postage stamp requests.
  • Fixed some problems with Destreak cleanup and update with regard to the camera stage masks.

Chris Waters

  • Fixed registration to set the state in rawExp and the data_state in rawImfile to 'full'
  • Tested and debugged software revision/mask statistics code for chip/cam/warp/stack/diff stages. Running simtest to track down any final issues before merging this back into the trunk. This took up much more of the week than I'd expected.
  • Began work on making JPEGs of warp/diff images on an exposure level.