PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : 2015-08-17

  • CZW: 15:20 Restart of ipp pantasks. I'm also going to try and find a host that can run the czarpoll scripts successfully.
  • CZW: 15:50 Between being distracted by other things, I've restarted czarpoll on ipp009.
  • CZW: 15:55 I also see that ipp061 is in nebulous state down despite the disks returning after the reboot. Set to repair.

Tuesday : 2015.08.18

  • 15:30 CZW: I'm going to restart the ippsky/pv3holes pantasks to attempt to flush out the last remaining full force stragglers. I'm using 1x loading on the m0, m1, x0, x0b, x1, x1b groups, as Bill suspects some of these may simply be memory issues.
  • 17:00 CZW: After testing, first set of convolved stack cleanup running in ippsky/clean_convolved_stacks. No logic in the choice of user.

Wednesday : 2015.08.19

  • 07:10 MEH: restarting pstamp
  • 10:40 MEH: ipp062 having xfs disk errors again, started ~10:33 HST -- neb-host down and like ipp061 taking out of regular processing
    • reboot -- /var/log/messages reports starting and ending recovery ok (no errors) so back to neb-repair
  • 12:40 MEH: Haydn going to take stsci19 offline to weigh it, neb-host down

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

  • 17:47 HAF restarts stdsci because it's needing it soon...

Saturday : YYYY.MM.DD

  • 12:25 MEH: Heather notes ipp061 inaccessible -- xfs errors again -- neb-host down
    • sync, reboot on console
    • stalled accessing homedir --
    • neb-host repair if reboot log indicates XFS starting/ending recovery ok -- seems ok

Sunday : YYYY.MM.DD