PS1 IPP Czar Logs for the week 2016.10.10 - 2016.10.16

(Up to PS1 IPP Czar Logs)

Monday : 2016.10.10

• 17:25 CZW: Restarting IPP pantasks servers.
• 18:10 MEH: as czar today... re-restarting pantasks w/ ipp092,091 removed in pantasks_hosts.input and 067 manually removed from processing and temporarily in neb-host repair along w/ 087,088 to adjust and evaluate for ipptopsps load like past 4 days (suspect 067 may be okay after talking with Heather)

Tuesday : 2016.10.11

• 10:20 MEH: Gene asked for other iffy nodes to just be automatically taken out of processing and neb-host repair to avoid conflict with PSPS file access spikes -- ipp067, 068 -- the daily czars will need to continue to monitor and adjust nightly processing as necessary for ~rest of week until P2 are finished (coordinate with Heather). maybe ipp086 out or just neb-host repair?
  • noticed ipp094 is still out from Gene's runs 20160729 -- this should be put back in to processing a while ago... (doesn't appear to have load issue but don't have time to check and leave for czar to deal with)
• 18:00 CZW: Restarting ipp pantasks.

Wednesday : 2016.10.12

• 17:15 CZW: Restarting pantasks.

Thursday : 2016.10.13

• MEH: bump up pstamp a bit for a large QUB request
• 17:00 CZW: stare03/04 had load spikes due to NFS issues mounting the home directory. These are also being used as test hosts for improving the Maui-ITC NFS connections.

Friday : 2016.10.14

Saturday : 2016.10.15

Sunday : 2016.10.16