Changes between Version 124 and Version 125 of PS1_IPP_Czarlog_20121203

Timestamp:

Dec 9, 2012, 1:52:38 AM (14 years ago)

Author:

Mark Huber

Comment:

--

PS1_IPP_Czarlog_20121203

@@ -160 +160 @@
   * rebooted, locks clearing. put into repair for a bit, may try and take out later
  * 22:59 CZW: noticed stacks were stuck.  It looks like the problem is that the stack.summary stage runs at a particular time each night, and launched ~100 jobs.  These jobs attempted to read data from stsci00, causing that host to grind to a halt trying to serve all the files.  This hasn't been a problem since we generally don't do 10k stacks a day.  I killed all the ppSkycell instances I could find, and that seems to have unstuck things.  I've disabled stack.summary for the weekend, and will turn it back on Monday with a lower number of concurrent jobs.
- * 01:20 MEH: registration stall for a while, lost file neb://ipp012.0/gpc1/20121209/o6270g0279o/o6270g0279o.ota16.fits. restarting registration
-=== Sunday : YYYY.MM.DD ===
+ 
+=== Sunday : 2012.12.09 ===
+ * 01:20 MEH: registration stall for a while, lost file neb://ipp012.0/gpc1/20121209/o6270g0278o/o6270g0278o.ota16.fits with a crash status.  restarting registration and running again.
+
+
+