PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : YYYY.MM.DD

• 12:29 MEH: adding CFA/MD07 WS publish for MOPS, restarting stdsci -- commented out until MOPS is re-setup to process

Tuesday : 2016.05.10

• 09:30 MEH: mysql gpc1 dump cleanup running on ipp001 so space isn't filled up..
• 10:11 MEH: sending to cleanup -- >1 month old WSdiff catalogs, ps_ud% (to clear the cycling faults) except the QUB targeted fields, ws_nightly_update, some QUB.% chips
  • leaving all the .multi so that cleanup can be done by others
• 11:35 MEH: 29/917G on homedisk -- starting bzip of pantasks logs, only freed ~5GB, other users need to manage their use more
• 11:41 MEH: ipp078 seems ok past day, neb-host up and see if error returns
  • @1345 noticed couldn't log in again -- console spammed with <May/11 07:22 am>[86659.748219] journal commit I/O error -- and stalls on INIT: Id "s0" respawning too fast: disabled for 5 minutes
• 17:34 MEH: restarted all nightly pantasks w/ ipp078 out of processing and neb-host down until can be looked into why behaving oddly

Wednesday : 2016.05.11

• 09:30 EAM: shutting down ippc01 - ippc16 excluding ippc11
• 10:20 EAM: czarpoll and roboczar services moved to ippc33 (see Processing)
• 13:20 CZW: Set ipp033, ipp035-ipp045 to down in nebulous, as they have had their nebulous data reinserted elsewhere.

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD