PS1 IPP Czar Logs for the week 2011-08-22 to 2011-08-28

(Up to PS1 IPP Czar Logs)

Monday : 2011-08-22

• 9:00 Bill shut down publishing, rcserver, and the postage stamp server to recover from out of disk space issue on ipppc17
• 12:20 Bill fixed a camera run with corrupt files (250162) that was causing some warps to fail
• 12:20 Bill fixed a corrupt skycell
• evening Bill queued diffs for STS.201108 19, 20, 21
• raised priority for STS and M31 nightlyscience to 400 to match the other labels.

Tuesday : 2011-08-23

• 00:00 Mark: made change to ppStack PSF matching rejection before new stacks ready and queued MD01.refstacks
• 7:40 Bill: added each of the stare nodes to stdscience twice
• 15:55 CZW: Restart stdscience to jumpstart processing after nebulous issue (still not fully understood).
• 17:50 CZW: Cycling power on ippc06 for the second time today. Removed from stdscience and stack processing.

Wednesday : 2011-08-24

• 02:15 Bill finds many faults on czartool. ipp014 load over 100. Unresponsive to ssh. Nothing on the console power cycled it.
• 02:57 ipp014 left a mess behind. Did force.umount on several hosts. On some it did not work. Removed stare01 and stare02 from stdscience for this reason
• 03:14 warpPendingSkyCell full of pages in state DONE.

Thursday : 2011-08-25

Friday : 2011-08-26

Saturday : 2011-08-27

Sunday : 2011-08-28