PS1 IPP Czar Logs for the week 2011.02.14 - 2011.02.20

(Up to PS1 IPP Czar Logs)

Monday : 2011.02.14

• 13:35 (heather) added magic test labels into distribution. All diffs/etc are done heather's stdsci, heather is preparing for the switchover to magictesting on the current trunk
• 16:40 (bill) Bill is running cleanup out of his build. The new code is working and has been checked into the production tree. Once last night's processing completes he will stop things update the production build and start cleanup from there.
• 16:40 (bill) dist.cleanup is off in order to give destreak cleanup time to catch up. The backlog is blocking postage stamp processing.

Tuesday : 2011.02.15

• 10:00 (eam) ipp005 is still rebuilding. I set it to neb down to protect it from activity. I tried to umount it, but could only remount it read-only.
• 11:09 (heather) heather added MD04.jtrp, MD05.jtrp, MD06.jtrp to ipp's stdscience. These have low priority. Heather also added (for diff of one item only) magictest.3Pi.ipp20101215.20110215. Heather is currently queueing/processing/kicking mopstestsuite.20110214. The processing for that mopstest goes into heather's stdscience, but the publishing will be done by ipp
• 17:30 (eam) stopped and restarted the stdscience pantasks, partly to clear out errors, but mostly to include the addnoise reduction class for the bigboss tests. (boss.MD*.20110215 labels)

Wednesday : 2011.02.16

• 11:45 bill queued 8 3pi exposures from 2010-12-30 for reprocessing. They didn't get run through magic then.
• 12:52 bill reset the warp book. It was full of entries in state DONE again.
• 16:05 bill added ThreePi.missing to distribution. This is 8 exposures from 12/30/2010 processed through stdscience this morning for Bertrand.
• 16:45 bill ran warptool -revertwarped on 3pi data. We have at least 1 corrupted file.

Thursday : 2011.02.17

• 18:28 bill turned dist.revert back on since ipp005 is back
• 21:00 bill restarted pstamp and update so that we don't have to look at the ridiculously large fault counts

Friday : 2011.02.18

• 12:40 CZW: Bill pointed out that ipp005 and ippdb00 both had loads approaching 150. This is probably due to the disk balance code still not being very well tuned. It seemed to have caused ippdb00/apache to go into the segfault loop. I stopped disk balance, restarted apache, and everything seems to be working fine now.

Saturday : 2011.02.19

Sunday : 2011.02.20