Changes between Version 203 and Version 204 of Processing

Timestamp:

Feb 19, 2016, 4:53:21 PM (10 years ago)

Author:

Mark Huber

Comment:

--

Processing

@@ -800 +800 @@
 
 * how to manually re-run for MOPS: http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/HowToManuallyRerunForMops
+
+= Pantasks Log Archive =
+
+(from Serge's notes somewhere, but is a czar responsibility)
+
+When restarting nightly pantasks with the start_server.sh, the pantasks logs get archived under a logs directory. These logs build up over time, particularly with regular unfixed errors/faults like in cleanup and can get 10-20GB in size over a few months. Without using the start_server.sh script they then grow to large size under their default names... 
+
+The logs need to be regularly compressed and archived
+
+ * on another (typically compute node) compress the logs under the pantasks directory -- may be slower but running on ippc19 will just harass that node computationally. For example stdscience
+
+{{{
+cd /data/ippc19.0/home/ipp/stdscience/logs
+bzip2 -pvf */*;bzip2 -pvf */*/*;bzip2 -pvf */*/*/*
+}}}
+
+ * since we are using ippc19 now as the active home node (late 2015/2016), the Archives are placed on ippc19.1 and a symlink Archives_c19.1 points to that location in the logs directory (previous use of ippc18 used the symlink Archives but is broken because the .1 mounts are not exported). Rsync from ippc19 and be sure to use bandwidth limits so not to overly harass ippc19...
+{{{
+rsync -avP --bwlimit=1000 201511 Archives_c19.1/
+}}}
+
+ * then can clean up the logs in the panstars directory (bzip2 will have helped clear up some space already) -- typically leave past/active month available to be able to quickly check logs for any recent problems.
+
+
+
+