PS1 IPP Czar Logs for the week 2014-03-03 - 2014-03-09

(Up to PS1 IPP Czar Logs)

Monday : YYYY.MM.DD

• 16:19 Bill ran add.sts.rp macro in stdscience to start up STS processing

Tuesday : 2014.03.04

mark is czar

• 06xx Bill, Gene rebooted ippdb01 and got nightly going again
• 06:50 MEH: ippdb01 down again, looks like root rebooted ~0700 -- looks like rebooted both times possibly missing a little RAM compared to earlier in week on ganglia
  • STS label out of stdsci until night finished.
• 07:20 MEH: czarpoll and roboczar stopped on ippc11 -- restarted
• 07:30 MEH: staticsky was stopped w/o a note, set back to run.. STS label back into stdsci
• 10:10 MEH: older STS throwing many fault 2 burntool issue -- chip.revert.off..

  burntool state vs burntoolStateGood : -13 vs 14
  Image burntool version does not match current accepted version. at /home/panstarrs/ipp/psconfig/ipp-20130712.lin64/bin/chip_imfile.pl line 830
  	main::my_die('Image burntool version does not match current accepted version.', 89358, 953683, 'XY01', 2) called at /home/panstarrs/ipp/psconfig/ipp-20130712.lin64/bin/chip_imfile.pl line 328
  

  • Bill has been changing label for these to STS.rp.2013.hold
• 10:20 MEH: ippc18.0 <70G.. doing log archive will free up ~30G only. apache node /tmp space >20G and okay.
• 11:50 MEH: noticed ipp004 has been constantly faulting stacks, was missing /local/ipp/tmp...

Wednesday : YYYY.MM.DD

Thursday : YYYY.MM.DD

• sometime -- Rita to do IPP port swap from 48 port switch to ippcore slot 9

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD