== PS1 IPP Czar Logs for the week 2017.03.13 - 2017.03.19 == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2017.03.13 ===
 * 21:00 EAM : we shutdown 1/3 of the cluster today to move to ITC.  I set up the pantasks host to avoid machines in the POD.  I also added in ipp121 and ipp078, which had been behaving themselves.  Unfortunately, around 20:00 ipp121 had a kernel panic and started to have nfs problems.  I rebooted ipp121 and took and ipp078 out of processing.
  
=== Tuesday : 2017.03.14 ===

 * 17:00 XZW: I've set up a pantasks to run rsync jobs between the low-numbered storage volumes and the high-numbered storage volumes (ipp118/119/120 for now).  These jobs should automatically stop at night (6:30PM), and restart in the morning (8AM), to ensure that nothing impacts nightly processing.  It will likely need additional tuning, as the server takes a long time to load the input file, but it is at least running.  Server directory /data/stare00.1/watersc1/itc_rsync_lists/rsync_run, running as ipptest.
=== Wednesday : 2017.03.15 ===

=== Thursday : 2017.03.16 ===

=== Friday : YYYY.MM.DD ===

=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===