Changes between Version 20 and Version 21 of PS1_IPP_Czarlog_20120716

Timestamp:

Jul 18, 2012, 1:02:35 PM (14 years ago)

Author:

Mark Huber

Comment:

--

PS1_IPP_Czarlog_20120716

@@ -25 +25 @@
  * 03:10 the 140 extra stack processing is hitting the stsci machines fairly hard, turning off 1x stare+2x compute+1x compute2 (~35) to start with. still causing some large spike, taking out 1x stare+1x compute2+1x wave4 more (~27).
  * 09:40 czar rate plot indicates ~300 stacks/hr with ~82 nodes. stsci nodes still getting spiked (nfs) but have handled for past few hrs.
- * 11:00 adding back 1x compute2+1x stare+1x wave4 for 103 nodes, stsci node spikes are broader now (time) -- monitoring
+ * 11:00 adding back 1x compute2+1x stare+1x wave4 for 103 nodes, stsci node spikes are broader now (time) -- monitoring -- seems okay and reaching >400/hr
  * 11:30 ganglia reports ipp008 all memory swapped but isnt. gmond restart worked for a little bit but back to saying all memory swapped. keep eye on.
  * 12:00 ecliptic.rp chip faults often NON-EXISTANT, but are being found with the new deneb-locate.py command -- details later
- * 12:50 pile up of ecliptic.rp in camera stage, 55ks runs
+ * 12:30 adding back 1x compute+1x compute2 for 124 nodes in stack pantasks 
+ * 12:50 pile up of ecliptic.rp in camera stage, ~15 runs >20ks. killing and reverting to see if clear ok.
+
 === Thursday : 2012.07.19 ===