IPP Progress Report for the week 2012.03.19 - 2012.03.23

(Up to IPP Progress Reports)

Eugene Magnier

I threaded the UpdateChips? portion of the relastro analysis -- this is the stage which does the main calculation work on a single machine, taking quite a lot of time (alternating with UpdateObjects?, which is quite fast). I was able to thread this code because the astrometry of all chips is independent, as long as the object positions are held fixed. The all-sky analysis (done in 13 chunks) was finally able to run in a reasonable amount of time (< 24 hours). I extracted a subset database for use as a reference catalog. I spent some time working on setting up code to generate summary plots of the photometry and astrometry residuals.

Serge Chastel

  • Sick on Thursday
  • Synced ipp001.0/gpc1 to ipp003.0/gpc1/ipp001
  • ippb02 intensive cares

Heather Flewelling

  • u band with JT
    • copied eamMC to megacam, updated the schema
    • turned off convolve for stacks for megacam
    • processed most MD fields by lunation for JT
    • helped JT modify his scripts to grab them (including problems on our end: ippuser now can access all the 'gpc1-like' databases (uip,isp, ssp, megacam)
  • cleaned up ippMonitor (removed projects that no longer have their associated databases)
  • some discussions with serge about nebulous
  • czar
  • restarted ThreePi?/LAP with new changes to addstar
    • new LAP: queued ssky and cam
    • few problems I need to sort out: there are timeouts for some of the tasks now: I need to both lengthen the timeouts and fix the queries that are doing that.

Roy Henderson

  • Sick on Monday
  • completed loading of all available LAP data
  • loading of MD4:
    • ingested full DVO database into MySQL
    • msqldump'd and restored onto other hosts in order to ingest for other clients
    • completed loaded in a few hours
  • time investigating dupe obj IDs: my fault - underestimated DVO access time (again)
  • improvements to program for setting up scratch Dbs
  • Object batches:
    • Dvo class has become an abstract base-class with two subclasses: DvoDetections? and DvoObjects?
    • both inherit a lot of common functionality, but are specific about what types of files are ingested and how
  • added some extra sections to the PSPS FAQ page

Mark Huber

  • Out sick on Wednesday.
  • Ongoing investigation into ppStack issues -- a larger sample with two skycells (055,058) in all grizy filters for MD04 produced two oddities in 055g,r that most of the week was spent on tracing out. 055g appears to be driven by the inputs to select a larger than desired target PSF while 055r appears to be a problem in creating or convolution to the target PSF.

Bill Sweeney

  • The postage stamp server got very busy this week. MPIA submitted about 300,000 stamp jobs. Spent quite a bit of time prioritizing things and making sure that their higher priority jobs got through. Spent some time adjusting the task timings and the various queue depths are not optimal for the situation where there are so many jobs outstanding when the cluster is not busy.
  • Staticsky: Did some more research into the large footprint problem. It turns out that these are being found when the unconvolved stack images are searched for detections. Analyzing the convolved images does not lead to this problem. Also investigated why psphot doesn't have this problem on warp stage images from busy areas. The answer is that the chip gaps in single exposure images prevent footprints from crossing chip boundaries. Setting footprint growth radius to 0 for pass to prevented the large footprints from being formed.
  • Staticsky for the recent LAP data, run with the improved background matching is nearly complete.
  • Investigated reports of astrometry failures near the pole. They turned out to be image quality problems, not astrometry problems.
  • Started working on a program to restore the uncensored camera stage outputs. Given the various data loses over the past couple of years this must be done carefully to avoid making data unavailable for update.
  • Generated chip stage images with the astrometric fit installed for the outreach program.

Chris Waters

  • False positive rate. Examined the false positive rate in the footprint data, and identified clear positional dependence of these false positives. Comparisons of true image variance against the variance maps shows the same position dependence, suggesting that the variance maps are not correctly modeling the image variance.
  • Constructed noisemap model. To add in the correct variance, I worked on adding a noisemap detrend type to the IPP configuration, and built one from bias frames. Preliminary results suggest that this creates a variance map that better models the true image variance, and as a result, we find fewer false positives.
  • Summary and plots can be found at https://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/Background_Dark_Model