PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : 2016.02.29

• 10:02 MEH: SNIaF updates running in normal stdscience (turn down/off as needed) and will be cleaned before nightly processing tonight
• 20:10 EAM: stopping and restarting pantasks for the night

Tuesday : 2016.03.01

• 06:00 EAM: cleared a stuck exposure in summitcopy (118o)
• 11:00 CZW: restart replication as ipptest on ippc05 for ippb03.1 ota data.
• 17:15 EAM: restarting pantasks

Wednesday : 2016.03.02

• 08:30 MEH: clearing fault 5 from 20160301 and last night before the accelerated cleanup get to them..
• 08:54 MEH: ipp087 BBU was fixed last week, it should have been restored as a data target host in ~ipp/psconfig/ipp-20141024.lin64/share/pantasks/modules/ipphosts.mhpcc.config...

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

• 15:00 EAM: stopping pantasks to adjust the processing lists (remove ipp070-ipp097 for now -- needed to run relastro)
• 15:15 EAM: I have adjusted the processing assignments to remove the _s5 set from the list (ipp070 - ipp097). I have replaced them with x2 and x3 for stdscience and s4 for summitcopy and registration (2 clients should not be an excessive additional load there).

Saturday : 2016.03.05

• 05:45 EAM: stdscience is far behind. not sure if this is due to the relastro load or not, but i'm restarting stdscience pantasks since it is not filling its queues.
  • update: it looks like ippdb05 is responding badly -- i'll quickly restart the mysql
  • stopping all ipp user pantasks to restart ippdb05 mysql
  • stopped addstar pantasks on ipp071, ipp072, ipp073, ipp074, ipp075, ipp076

Sunday : YYYY.MM.DD