PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : 2011.11.14

• 08:40 Mark: looks like ippc06 is down, seems to be causing problem even though out of nebulous hosts? maybe problem in my config (yes, see ~ipp/.tcshrc). rebooting (no info on console)

  Unable to access file neb://@HOST@.0/gpc1/SAS2.123/2011/07/26/RINGS.V3/skycell.1405.001/RINGS.V3.skycell.1405.001.stk.339528.target.psf: nebclient.c:535 nebFind() - no instances found
  
  -- but neb-stat says ok
        1 409a428a70bcbbccd1528cecdd33c02a file:///data/ipp037.0/nebulous/f6/92/1132632500.gpc1:SAS2.123:2011:07:26:RINGS.V3:skycell.1405.001:RINGS.V3.skycell.1405.001.stk.339528.target.psf
        1 409a428a70bcbbccd1528cecdd33c02a file:///data/ipp027.0/nebulous/f6/92/1132694136.gpc1:SAS2.123:2011:07:26:RINGS.V3:skycell.1405.001:RINGS.V3.skycell.1405.001.stk.339528.target.psf
  
  

• 14:50 Mark: stress testing CPU on ippc11 to check if can use again after heatsink reset. CPU1 usage (up to 4 cores) may be possible, full 8 cores lasted 3 minutes. Also isn't rebooting on powercycle again, leaving off for a bit and back after ~5-10 mins (seems like thermal issue still).
• 16:16 Mark: crashed ipp029 with same stress code (only 2 jobs) and got kernel panic like has been seen before, probabaly not thermal issue.
• 16:30 Mark: crashed ippc11 again, 6 jobs running. temperature plots show temperature rise as expected, then drop off after some 15 mins, then spike one CPU and crash. suspect ippc11 could now be used like ippc13 in the hosts_poor_compute group, at about 30-50%.

Tuesday : YYYY.MM.DD

Wednesday : YYYY.MM.DD

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD