IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:PS1_IPP_Czarlog_20160404

Version 7 (modified by Mark Huber, 10 years ago) ( diff )

--

PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : 2016.04.04

  • 09:34 MEH: more SNIaF updates in ippqub:stdscience_ws
  • 20:20 EAM : stopping pantasks for restart

Tuesday : YYYY.MM.DD

Wednesday : 2016.04.06

  • MEH: gpc1 hasn't had a proper dump since ~2/4 when ippdb03 had OS disk raid issues and had to be swapped with ippc15 and was not checked...
    • /var/spool/cron/crontabs group stb-admin not crontab.. fixed so now ipp has crontab access again on ippdb03...
    • re-creating crontab best can to follow info on DatabaseBackups as best possible..
  • MEH: ippMonitor has been reporting problems (red) with ippdb03 (gpc1 replication) and ippdb06 (nebulous replication) for some time now with no one looking into -- this dates back to the swap of ippMonitor to another machine while ippdb03 being remade -- there are all kinds of inconsistencies in host names, passwords in ippMonitor AND mysql setups... tweaked to get ippMonitor to report on those so we can be alerted to problems now.. but cannot easily clean up inconsistencies as who knows what all rely on them as things are not documented..

  • 15:45 MEH: Haydn replacing BBU in ipp085 -- all pantasks stop for now
  • 17:40 MEH: restarting nightly pantasks with data targeting adjustment off the nodes with less free space

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD

Note: See TracWiki for help on using the wiki.