Changes between Version 14 and Version 15 of PS1_IPP_Czarlog_20190204

Timestamp:

Feb 6, 2019, 5:12:09 PM (8 years ago)

Author:

Mark Huber

Comment:

--

PS1_IPP_Czarlog_20190204

@@ -65 +65 @@
 }}}
 
- * MEH: increased ippdb08 mysql from 256 to 512 for tonight with set global (ephemeral) and not in my.cnf to monitor how things go
-
 JRF: The stdscience log last night on PS1 showed a bunch of errors reported around 20:30. The majority of these were on ippc118. Look at the ganglia logs there was a spike in network activity, up to about 17Mb, but not too large that it should cause an issue. Either way the tasks that failed reverted successfully and continued on after this.
 
@@ -80 +78 @@
 }}}
 Commented out the use of gpc2 dates in the 'stdscience/input' file. Believe this is the reason that there are gpc2 errors being reported in the gpc1 logs above.
+
+MEH: the logs have useful info for tracking down problems and while the ~ippitc/start_server.sh script archives the logs, it might also be good to keep a note of running tasks and taskstats before doing a shutdown -- one way can be like this
+{{{
+echo "status;quit" | pantasks_client -c ~ippitc/stdscience/ptolemy.rc >& stdsci_status.`date +%y%m%d_%H%M%S`
+echo "status -taskstats;quit" | pantasks_client -c ~ippitc/stdscience/ptolemy.rc >& stdsci_tasksstats.`date +%y%m%d_%H%M%S`
+echo "controller status;quit" | pantasks_client -c ~ippitc/stdscience/ptolemy.rc >& stdsci_nodes.`date +%y%m%d_%H%M%S`
+}}}
+
+MEH: increased ippdb08 mysql from 256 to 512 for tonight with set global (ephemeral) and not in my.cnf to monitor how things go
 
 JRF: killed a 4 day old job in the database that was run as ippuser to query something. WARNING: very dangerous to go around killing jobs.