== PS1 IPP Czar Logs for the week 2016.10.03 - 2016.10.09 ==
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2016.10.03 ===

 * 16:30 HAF daily restart of panstasks
=== Tuesday : 2016.10.04 ===

  * 16:10 HAF daily restart of pantasks
=== Wednesday : 2016.10.05 ===

=== Thursday : 2016.10.06 ===
 * MEH: apparently ganglia on ippdb01 (nebulous) hasn't been reporting for several months? gmond status shows running, but doing restart and now ganglia reporting status again
  * also notice similar issue w/ ipp017 non-reporting for past couple weeks, running again
 * 17:40 EAM : restarting pantasks
 * 21:55 MEH: ipp068 seems to be having trouble -- take out of pantasks and neb-host repair for now
=== Friday : 2016.10.07 ===
 * MEH: good number of data nodes going red, doing misc cleanup where possible -- ps_ud_%, sniaf updates, QUB.WS not needed, 
 * MEH: removing nodes following from processing and setting repair to try an avoid stalls -- ipp092, 067, 091, 054
=== Saturday : 2016.10.08 ===
 MEH: restarting pantasks w/ ipp092,091,067 manually removed from processing and pstamp tweak bump again to help clear large stamp request
=== Sunday : 2016.10.09 ===