IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:PS1_IPP_Czarlog_20141020

Version 21 (modified by eugene, 12 years ago) ( diff )

--

PS1 IPP Czar Logs for the week 2014.10.20 - 2014.10.26

(Up to PS1 IPP Czar Logs)

Monday : 2014.10.20

  • 06:00 Bill set pstamp pantasks to stop to prepare for periodic restart
    • 6:19 restarted
  • 18:50 MEH: ippsXX off in lanl stdlocal -- using for MOPS test chunk reprocess (ippmops@ipps01 client)
  • 20:30 MEH: someone restarted lanl stdlocal and left ippsXX on.. noted here and in email, can't be more informative that that..
  • 23:55 MEH: returning ippsXX

Tuesday : 2014.10.21

  • 16:25 MEH: ippsXX used for MOPS test reprocessing again -- off in ippsXX. if you restart lanl stdlocal please be sure to turn them off before setting run...
  • 21:00 MEH: turning ippsXX back on in lanl stdlocal
  • 21:06 EAM: restarting stdlocal

Wednesday : 2014.10.22

  • 12:40 CZW: restarting stdlanl to see if that will sort out the poll task not polling as much as it could.
  • 20:30 EAM: restarting stdlocal

Thursday : 2014.10.23

  • 20:30 MEH: notice processing slowing and stack turned off in lanl stdlocal, guessing someone is going to do a regular restart so won't touch
  • 20:55 EAM: yes, restarting in just a moment...
  • 21:20 EAM: restart complete.

Friday : 2014.10.24

  • 10:00 MEH: will start the restart of all nightly processing pantasks in prep for possible observations tonight
  • 11:00 MEH: ippsXX out of lanl stdlocal for MOPS diffim reprocessing --
  • 12:20 MEH: ippsXX back in lanl stdlocal
  • 14:20 MEH: lanl stdlocal not running stacks, looks like could use regular restart
  • 14:27 EAM: stopping and restarting stdlocal. I will set up stdlocal to block if too many nightlyscience jobs are outstanding.
  • 14:35 EAM: I've turned on the tasks which remove storage host for the night (20:00 HST - 05:30 HST) to avoid overloading. Since I also have the code on to block if too many nightly science jobs are outstanding, this is probably a bit conservative (thus the restrictive hours).
  • 14:42 EAM: I've also paused 2000 chipRuns for lap.pv3.20140730.ipp -- the ippdb01 load may partly be from the 3000 chips ready to run.

Saturday : 2014.10.25

Sunday : 2014.10.26

Note: See TracWiki for help on using the wiki.