PS1 IPP Czar Logs for the week 2017.05.29 - 2017.06.04

(Up to PS1 IPP Czar Logs)

Monday : YYYY.MM.DD

Tuesday : YYYY.MM.DD

Wednesday : 2017.05.31

  • 04:30 EAM : mysql on ipp106 filled up /tmp; I cleared binlogs to get things running.
  • 16:10 CZW: Restarting ippitc pantasks.
  • 17:50 CZW: Adding x3 nodes (ippx076-ippx088) to HSC pantasks (/data/stare04.1/watersc1/hsc/stdsci). I don't think these have been used for anything since the move. This is chip+camera processing only right now.

Thursday : 2017-06-01

  • 11:20 CZW: Rebooting stare03 in an attempt to resolve NFS issues that have only gotten worse. I may end up rebooting all of the stare nodes if needed.
  • 11:54 CZW: Rebooting stare04 to clear NFS. Rebooting stare01 while I'm at it, as it has the missing memory.
  • 12:30 CZW: stare04 refuses to come back online with all memory visible. I'm leaving it up and accepting that only 48667508 / 97425940 will be available. There is no power control, so I can't see if turning it off for a bit will sort things out.
  • 16:15 CZW: Restarting ippitc pantasks.

Friday : 2017.06.02

  • MEH: queuing up the SNIaF cals from last night in normal nightly processing
  • MEH: running more MOPS tests on ipps nodes

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD