== PS1 IPP Czar Logs for the week 2017.05.29 - 2017.06.04 == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : YYYY.MM.DD ===
  
=== Tuesday : YYYY.MM.DD ===

=== Wednesday : 2017.05.31 ===
 * 04:30 EAM : mysql on ipp106 filled up /tmp; I cleared binlogs to get things running.
 * 16:10 CZW: Restarting ippitc pantasks.
 * 17:50 CZW: Adding x3 nodes (ippx076-ippx088) to HSC pantasks (/data/stare04.1/watersc1/hsc/stdsci).  I don't think these have been used for anything since the move.  This is chip+camera processing only right now.
 
=== Thursday : 2017-06-01 ===
 * 11:20 CZW: Rebooting stare03 in an attempt to resolve NFS issues that have only gotten worse.  I may end up rebooting all of the stare nodes if needed.
 * 11:54 CZW: Rebooting stare04 to clear NFS.  Rebooting stare01 while I'm at it, as it has the missing memory.
 * 12:30 CZW: stare04 refuses to come back online with all memory visible.  I'm leaving it up and accepting that only 48667508 / 97425940 will be available.  There is no power control, so I can't see if turning it off for a bit will sort things out.
 * 16:15 CZW: Restarting ippitc pantasks.  
=== Friday : 2017.06.02 ===
 * MEH: queuing up the SNIaF cals from last night in normal nightly processing
 * MEH: running more MOPS tests on ipps nodes 
=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===