== PS1 IPP Czar Logs for the week 2017.07.10 - 2017.07.16 == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2017.07.10 ===
 * MEH: as discussed at the meeting last week and this week, priority w-band refstacks need to be running on as many nodes as possible, but are removed from data nodes during nightly processing (~ippqub/stack/ptolemy.rc, PSNSC.wref.20170707)
  * ipp058-066 repair->up during day for refstack processing (until h02-con power management setup)
  
=== Tuesday : 2017.07.11 ===
 * 13:00 EAM : I have updated czarpoll for ippMonitor in a couple of useful ways:
   * I have fixed a bug preventing it from reading the labels from the pantasks
   * I have modified the code to merge the stdscience and distribution pantasks labels together for display 
     * this latter should obviate the need to manually hack the CzarDb.pm code to add in labels.
   * I have added the gnuplot binary code path to the .xml config file, and removed the hard-coded GNUPLOT value from Plotter.pm
   * I have moved the config file up one level (to ippMonitor/czartool/czarconfig.xml from ippMonitor/czartool/czartool/czarconfig.xml) so it is not so hidden
 * MEH: adding limited w refstack processing on ippx nodes along side dvopsps processing -- x051-x064
 
=== Wednesday : 2017.07.12 ===
 * MEH: cleaned up misc K2+MD07 nightly products 
 * MEH: continuing to add limited refstack processing on ippx nodes along side dvopsps processing -- x030-x050
 * MEH: ipp121 ganglia down, restarted gmond and reporting again so any issues can be logged...
 * 16:00 CZW: power cycled stare01, which I seem to have killed accidentally.
=== Thursday : 2017.07.13 ===
 * 06:50 MEH: a very long list of summitcopy fault 200 -- using revertcopy to clear
{{{
request failed: 500 EOF when chunk header expected at /data/ippc64.1/ippitc/psconfig/ipp-20170121.lin64/bin/dsget line 155.
}}}
 * 07:20 MEH: lost connection to summit for a few mins at least
{{{
request failed: 503 Service Temporarily Unavailable at /data/ippc64.1/ippitc/psconfig/ipp-20170121.lin64/bin/dsget line 155.
}}}
 * MEH: network issue summit faulted files before manually started revertcopy -- manually set fault 0 in gpc1 to re-download
{{{
7 summit faults
  exp_name         registered      fault
o7947g0105o   2017-07-13 14:44:34   203
o7947g0107o   2017-07-13 14:46:03   203
o7947g0108o   2017-07-13 14:46:44   203
o7947g0109o   2017-07-13 14:47:29   203
o7947g0110o   2017-07-13 14:48:10   203
o7947g0111o   2017-07-13 14:48:51   203
o7947g0116o   2017-07-13 14:52:28   203
}}}
=== Friday : YYYY.MM.DD ===

=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===