IPP Progress Report for the week YYYY.MM.DD - YYYY.MM.DD

(Up to IPP Progress Reports)

Eugene Magnier

Serge Chastel

• Fixed 13902 out of 16759 (about 83%) 0-size instances "on ippb02" (as detailed during the ipp meeting: the instances have been actually moved to ippb03). Some faulty instances on other hosts have been fixed as well (0-size files and wrong md5sum according to [gpc1|isp]/pzDownloadImfile tables). The complete list of the files that have been fixed can be found in ~ipp/sch/fix_ippb02/repair/good. All instances that have been fixed had a good copy somewhere on the cluster.
• I'm now checking the 259 "possibly repairable" instances, that is those for which the size and md5sum are not set in the [gpc1|isp]/pzDownloadImfile tables.
• There are 2598 instances that cannot be repaired: list in ~/sch/fix_ippb02/repair/others/cant_fix.txt. I need to make more work on those though. Some instances might be lost in nebulous but could still be on one of the disks.
• Once all faulty files on ippb02 have a known status (good or bad) I will perform a new overall check to be sure that everything's fine on it from the raw images point of view. I will then *) start looking at the other products on ippb02; *) make sure there are exactly two instances of the raw images and that 1 is at mhpcc, 1 at atrc *) remove the raw images which have in more than 2 copies.

Heather Flewelling

Roy Henderson

• sick for 3 days
• figured out how to queue up object batches using unique DVO region IDs
• re-factored metrics and loading code to accommodate the above
• create, mostly empty, OB batches for testing on the PSPS side
• reading in Photcodes.dat from DVO in order to get to mags for each objID
• started work on stored procedure to deal with DVO object mags stored by row number rather than ID

Mark Huber

• ppStack continuing to trace out the two remaining example problems of larger then expected target/final convolved stack PSF for making deep stacks.
• Czar Friday and personal research/telescope proposal.

Bill Sweeney

• Performed some experiments to see how expensive it would be to regenerate camera stage mask files that have been lost. With the current code we can limit the chips that psastro operates on but the resulting masks are different. Specifically glints from other chips on the camera are not included. To get them the full analysis is required which takes several minutes because it redoes the astrometry.
• Finished destreak_camera_restore script which will recover space used by censored smf and mask files. Started running it over the weekend. It takes about 200 seconds per camRun so it will take a few weeks to complete. (We don't want to assign too many hosts at once to work on this because it can be stressful to nebulous)
• psphotStack still can't deal with the galactic plane (at least near the center). Wrote a script to limit the runs to with abs(galactic latitude) > 20 for now.
• 2 days as processing czar. One day spent several hours finding bottlenecks in processing. It turned out to be the galactic plane again. pswarp could use some work reducing memory footprint. Also removed the nodes running psphotStack from stdscience so that memory bloat doesn't block that

processing.

Chris Waters