== PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : YYYY.MM.DD ===
  * 16:30 HAF: mystery czar restarted pantasks for today. Thanks!

=== Tuesday : YYYY.MM.DD ===

=== Wednesday : 2016.06.01 ===
 * ippx026 crashed ~0130 -- Haydn power cycled on-site. Reminder, power cycle via console will power cycle to 4 nodes associated with the PDU so use caution when doing that.
 * Haydn working on ippc nodes replacing disks -- setting off in processing until finished
  * ippc54.1 unmounted and presumed lost? Hadyn replacing disk and remaking raid
  * ippc60.1 another disk needed replacing, no new data there since last disk replaced so done and remaking raid
  * ippc43.0,c57.0 non-destructive disk replacement (rebuilding)
  * MEH: ippc54.1 unmounted and suspect missing since early 2016 by note in pantasks_hosts.input -- uncommenting for c2 group now that c54.1 disk is back
 * MEH: several ippc nodes missing .1/ipp/tmp dir that is linked from /local/ipp/tmp -- explains some nightly processing faults been having -- restoring those tmp dirs for processing on c32,c39,c40,c43,c54,c56,c60
 * MEH: ippc55,c62 seem to be commented out with NO note.. seem ok, so uncomment and watch in tonight's processing...

=== Thursday : YYYY.MM.DD ===

=== Friday : YYYY.MM.DD ===

=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===