IPP Progress Report for the week 2010.08.02 - 2010.08.06

(Up to IPP Progress Reports)

Eugene Magnier

I have been looking more at the difference image analysis, and have identified and fixed a couple of issues.

The first problem was related to the normalization needed to match the flux in the two images, a prerequisite to matching the PSFs. This is essentially relative aperture photometry on both of the images. However, the apertures were defined to be the same for both images; as the images had very different image qualities, the apertures measured different fractions of the total flux, resulting in a bias. I have fixed this by defining separate apertures for each image, and at the same time I've made the aperture size a bit more robustly defined in terms of an effective curve of growth.

The second issue was specifically related to dual convolution. In the dual convolution process, we are attempting to find the convolution kernels kA and kB which together match a pair of images A and B. This problem is over-determined: we can always add something to both kA and kB to increase the size of the convolved image. To minimize this, we have a penalty function which penalizes large kernels relative to small kernels (following Yuan & Akerlof). The penalty function involves the second moments. However, it was defined based on the second moments of the kernel set, resulting in the same values for both images. Instead, the image with the larger PSF should have larger penalties for the same sized kernels (ie, we should tend to smooth the smaller image). I've fixed this by adding the second moments of the images in quadrature to the kernel second moments; this seems to have the desired effect.

Serge Chastel

  • IPP
    • Two and a half days spent on addtool_find_cam_id SQL query. The query is now executed (almost) instantly whereas it required 30 minutes before.
    • Czar on Thursday and Friday (Thanks to all IPP team members for their support)
  • PSPS
    • I wrote some code to access/query the PSPS database through web01. The code was written in Python since we want something independent from existing PSPS software that can be used for testing activities. The next step is FITS reading and then match FITS file contents against database ones. Note: Python is an arbitrary choice but the language that will be used for testing activities has to be defined.
  • Cancelled my participation to Belfast workshop because of my passport concerns

Heather Flewelling

  • description of cmf/smf columns (with help from Roy and Gene): http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/cmf_columns
  • attended 3 hours of meetings
  • IPP-MOPS liason
    • helped the quattro italiano guys get accounts, access ipp, access fits files
    • transferred a GOOD and a BAD diff and warps for them to view
    • investigated and catalogued detections of one of the skycells on the BAD diff.
  • ifaps1 liason
    • answered questions
    • had gavin set up mount point to ipp022 for dvo catalog
  • czar for 2 days
    • dropped exposures with filter = "not available", as per Bill's email
    • cleaned up some of the faults on ippMonitor. There were many, I was only able to unstick some of them.
    • helped serge on his czaring, also showed serge and roy how to unstick the unrevertables (one of the easier ways - revert faults !=2)
    • queued stack stack diffs for MD08
  • got my passport with my correct name.

Roy Henderson

  • PSPS
    • Tracked down problem with data given to Jim last week: my fault (one frame loaded multiple times). Fixed.
    • Attempts to load to Maui with new DXLayer, various problems:
      • wrong paths in config
      • Invalid format specification error: investigated
      • wrong permissions on sandbox so could not debug there
  • IPP
    • Czar on Monday:
      • registration down, restarted
      • high load on ipp45: killed distribution, seemed to help
      • MD08 stack-stack diff. Tutorial from Heather
      • updates to czar wiki page, specifically, the reverts section
    • czartool:
      • revert status now written to Db from czartool so status on page is accurate at all times
      • added Nebulous class. roboczar now using this to log cluster disk use.
      • creating plot of cluster space over all hosts; added to czartool page
    • ippToPsps:
      • re-wrote exposure-getting SQL to only get exposures that are in the provided DVO Db and to pull out correct relevant smf files
      • added datastore product, DVO db and min and max object IDs to database table for auditing purposes

Bill Sweeney

  • Spent most of this week working on dvo issues.
    • enhance dvomerge to allow catdirs with different photcode definitions to be merged properly.
    • fixed some errors in the sdss import that were discovered while working on dvomerge
  • Implemented and tested new survey tasks for the new background stages.
  • fixed bug in bgtool exposed by the survey task.
  • Investigated the cause of recently degraded data transfer speeds achieved by consortium members. Tracked the problem down to an overloaded web proxy.
  • investigated failed detectability query requests. Transferred information learned on to Chris.
  • did some database queries to measure the number of "unfinished exposures" that is data taken but not yet run through magic.
  • many hours spent debugging processing faults and other issues related to processing.

Chris Waters

  • Two days vacation.
  • Linearity: worked on a way to fix the non-linearity results in software, then discovered that the data may be confused due to dark processing. Will redo with no dark application to confirm nonlinearity. Queued a third set of data to check repeatability.
  • Automation: reports of missing stacks pointed out a bug in the automation code. When the camera stage astrometry fails (as it did this week due to loss of guide stars), the automation code doesn't notice and waits for them to finish warping. Will fix next time I edit the code.