Changes between Version 129 and Version 130 of PS1_IPP_Czarlog_20141201

Timestamp:

Dec 6, 2014, 10:52:46 PM (12 years ago)

Author:

Mark Huber

Comment:

--

PS1_IPP_Czarlog_20141201

@@ -205 +205 @@
   * ippc29 missing memory again -- i will power cycle later, only a test/stack node for MD
   * datanode mounting datanodes seems ok, c0-2,m0,m1 mount of datanodes seem ok --  
-
  * 16:05 MEH: limited nightly processing started since uses data nodes, some misc compute nodes have to be looked at still
   * exposure blocked in summitcopy o6997g0719o -- others getting stalled but slowly clearing, restarting summitcopy cleared
@@ -211 +210 @@
   * NO other processing should start until nightly finishes, then will start cleanup and pstamp 
   * ippc19 date is wrong, ippc07 having trouble mounting homedir -- fixed w/ manual time set and restart ntpd, restart nfs
-
  * 17:45 MEH: nightly caught up except for darks, starting other pantasks
   * adding back WS labels back in for diffims -- survey.warpstack.diff repeat timeouts, gpc1 having trouble 
   * looks like slow query is pilling up and not clearing, may need to restart db01 mysql -- still happening, will have to turn off WS diff or will drive db into the ground
- 
  * 18:50 MEH: the long WS and unused stack nightlyscience query faults after ~600s but continue to pile up in the db -- disable WS diffs and stack tasks..
   * will need to try and get the WS diff caught up by monday to avoid cleanup problem?
 
- * 20:30 MEH: watching ippmd + ipplocal running w/ stdsci until first full chunk done -- no diffims made so tied to ns.stack ?
+ * 20:30 MEH: watching ippmd + ipplocal running w/ stdsci until first full chunk done -- seeing no diffims being made..
   * '''if starting other pantasks, please at least note it in czarlogs...'''