PS1 IPP Czar Logs for the week 2011-05-09 - 2011-05-15

(Up to PS1 IPP Czar Logs)

Monday : 2011-05-09

Bill is acting czar today.

• 15:18 incomplete list of activities done so far today
  • restarted distribution with double the number of nodes (since it was way behind)
  • moved warp to it's own pantasks.
  • Now we have a few too many pclients running on some of the weaker machines. Some nodes have entered swap heck. So removed 2 of the 3 host entries in distribution for ipp013 and ipp037.
  • Created a new publishing client and associated data store IPP-MOPS-LAP
  • queued next set of runs for STS.2010.b (STS.b.20100831)
  • repaired 2 warp and one diff skycells that had corrupted files. (ipp050, ipp051, and ipp025)

Tuesday : 2011-05-10

Bill is acting czar today.

• Still getting lots of faults and file corruption. A couple of magicRun's got very damaged with zero sized streakMap files. Our tools aren't ready for this.
• 11:51 Stopping stdscience. The warp pantasks experiment is causing too much load on the system.
• 12:33 stdscience restarted including warp.

Wednesday : 2011-05-11

• Bill: 09:40 Still getting some file corruption. chip 229883 XY04 warp 196230.skycell.1198.051 Fixed with tools/run*
• 09:45 turned off chip processing to allow the nightly science warps to finish faster

Thursday : 2011-05-12 ==

Friday : 2011-05-13

Saturday : 2011-05-14

Sunday : 2011-05-15