Changes between Version 16 and Version 17 of PS1_IPP_Czarlog_20140210

Timestamp:

Feb 12, 2014, 12:25:20 PM (12 years ago)

Author:

watersc1

Comment:

--

PS1_IPP_Czarlog_20140210

@@ -16 +16 @@
  * 14:00 Bill: removed right ascension limit on staticsky so that all pending jobs will run. Once this set clears out we'll start queueing new ones and when I do that I'll order them by slices.
  * 15:00 CZW: restarted all pantasks for the daily restart.  This should pull the new pantasks.pro code that properly removes CRASH-ed job pages.
-=== Wednesday : YYYY.MM.DD ===
-
+=== Wednesday : 2014-02-12 ===
+ * 12:20 CZW: I just noticed that we have a very large number of chips/warps in the processing queue.  There are now 274 lapRuns active, up significantly from the ~75 a few days ago.  It appears that the cleanup task is timing out for the one run still in 'full' state, and when it times out, it doesn't correctly clear out the full run, leading to many more lapRuns being launched that is expected.  I've pointed ~ipp/lap/current.queue to the empty off.queue, which will prevent new runs from being launched (so we can taper back down to ~75).  I'll manually clear the full run, and figure out how long to set the timeout limit to.
 === Thursday : YYYY.MM.DD ===