(Up to PS1 IPP Czar Logs)

Monday : 2012.10.08

Mark is czar

• 00:10 MEH: stdscience struggling to keep loaded >50%, needs regular restart
• 07:20 MEH: some 580 nightly science exposures finished downloading, processing ongoing.
• 07:50 nightly science done, chip.off to force warps through and load LAP stacks and set.poll 400-600 slowly to keep all nodes possibly loaded but not overload the stsci nodes at once (remember to do again later afternoon to have LAP stacks loaded for night).
• 09:00 Serge fixed replication to ippdb03, ipp001 Replication_Issues -- czar page slowly catching up
• 09:50 MEH: fixing LAP chips holding up warps->stacks

  -- gone
  neb://ipp027.0/gpc1/20100901/o5440g0299o/o5440g0299o.ota26.fits
  -- found orphan
  neb://ipp044.0/gpc1/20100820/o5428g0097o/o5428g0097o.ota31.fits
  neb://ipp029.0/gpc1/20100731/o5408g0328o/o5408g0328o.ota33.fits
  neb://ipp041.0/gpc1/20100924/o5463g0316o/o5463g0316o.ota21.fits
  neb://ipp040.0/gpc1/20100518/o5334g0358o/o5334g0358o.ota14.burn.tbl
  neb://ipp020.0/gpc1/20100714/o5391g0128o/o5391g0128o.ota40.fits
  neb://ipp029.0/gpc1/20100814/o5422g0123o/o5422g0123o.ota33.fits
  -- remade
  neb://ipp040.0/gpc1/20100518/o5334g0369o/o5334g0369o.ota15.burn.tbl
  -- dropped exposure from 8/7/2012 in LAP 8745, was in odd state maybe from cleanup on/off
  laptool -updateexp -lap_id 8745 -exp_id 506605 -set_data_state drop -dbname gpc1
  

• 14:00 replication of ippdb03 caught up now. ipp001 caught up around noon. turning chip.on again as well for LAP
• 14:50 nearing time for stdscience restart again.. done
• 15:10 should check on devel cluster machines occasionally to make sure temps are getting out of hand -- ipp003 temp1/2 idle @44,50F

Tuesday : YYYY.MM.DD

Wednesday : YYYY.MM.DD

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD