Changes between Version 70 and Version 71 of PS1_IPP_Czarlog_20150112

Timestamp:

Jan 17, 2015, 9:03:54 AM (12 years ago)

Author:

Mark Huber

Comment:

--

PS1_IPP_Czarlog_20150112

@@ -147 +147 @@
  * 15:30 EAM: Haydn rebooted ipp083 & ipp086 to fix raid betteries.
  * 16:40 MEH: ipp084, ipp091 have BBU and in WB state -- neb-host up now, will wait to add to data host targeting when the rest go online as well (083,086,092), so same targeting as last night
- * 21:20 MEH: are the QUB stamps stalled? going to restart pstamp -- still seems so
+ * 21:20 MEH: are the QUB stamps stalled? going to restart pstamp -- still seems so -- looks like some updates with label for ps_ud_QUB were then cleaned
  * 23:10 MEH: bumped stdsci up 5x:s4+s5 -- seems to keep up. ipp067-095 have a fairly high load (cpu~60%), summitcopy+registration+stdsci fine, but like previous nights can have load spikes ~50 and some wait%.
  * 23:20 MEH: OSS warp fault to quality clear  -- cannot build growth curve (psf model is invalid everywhere)
@@ -162 +162 @@
  * 05:25 EAM: looks like things ran pretty well last night - we are in pretty good shape on the nightly processing and stdlocal ran solidly at 75-100 chips/hour.  I notice that the following stages are lagging in stdlocal, and have attributed this to a backup in camera.  basically, the galactic plane is slower in camera because it scales by number of stars (even more so than chip).  since we have a fairly limited number of camera jobs at a time, this becomes a bottleneck.  I've bumped camera polling from 25 to 60 to address this, but it perhaps could go higher.
 
+ * 09:00 MEH: going to try and fix the cleanup/update conflict blocking QUB stamps..
+
 === Sunday : YYYY.MM.DD ===