IPP Progress Report for the week 2010.03.22 - 2010.03.26

(Up to IPP Progress Reports)

Eugene Magnier

Heather Flewelling

Bill Giebink

Roy Henderson

• PSPS
  • Data loading
    • Results of test loading: looked into errors from test batches: a duplicate object ID and some 'inf' values causing problems
    • Finally got an Object batch loaded, so now we've tested initialisation, stack, detection and object
  • Other
    • Worries about compressed binary data needed for diffs: did some research, met with Conrad and Jim to discuss
    • Started work on modularising DXLayer, specifically the a fits2csv chunk

• IPP development
  • pspsSchema2xml script now generates C headers with enums for PSPS tables.
  • Started changes to ippToPsps to accommodate new PSPS schema

• Other stuff
  • Dentist on Thursday

Paul Price

• Cluster cleanup: destreaked warps, diff convolved images
• Added 3 kinds of exposure maps to ppStack (number, exposure time, weighted exposure time). Set static BSCALE,BZERO so these are not too large.
• Updated ppCoords so it can calculate cell coordinates in addition to chip coordinates
• Set up system to collate microtest magic clusters in cell coordinates; published data to Camera group.
• Investigating missing skycells in warp-warp diffs:
  • Found that warp-warp diffs don't always include all skycells available.
  • One possible cause is use of '-available' flag with difftool -definewarpwarp. Removed this flag: no longer necessary, and bad for current pipeline operating model. Suspect another cause as well.
  • Almost all warp-warp diffs have been affected, at a typical level of 10% of skycells missing. No, turns out this 10% are edge skycells, ignored due to constraints on good_frac.
  • Warp quality flags are all reasonable, and represent different stages of skycell overlap, from none to some with no sources, to some sources with no good PSF, to good.
  • Bad quality warps extend into the centre --- should have good PSF measurement there, but it's failing.
  • The SMF files are corrupted, with many extensions empty. Dave Monet had told me about this, but I hadn't realised how extensive it is. I have fixed psastro (was not checking return values) previously, so this should be a solved problem, but we're now dealing with its effects. Hopefully, that means that the problem is not so extensive as we thought.
  • Sifting through SMF files to identify bad files.

Bill Sweeney

Chris Waters

• Diskspace: Finish up monitoring raw Imfile compression for engineering data. Will probably need to flag more images for compression soon.
• Detrends: Wrote script to bundle detrend images for distribution. First set done and posted, and the remainder are being bundled.
• Processing czar tasks.
• Detectability: Work to correctly respond to a detectability query from the postage stamp server.