PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : 2013.03.18

mark is czar

• 01:00 MEH: putting 1x compute3 back on in deepstack for the staticsky outside of the GP for now. leaving normal allocation of 1x compute3 in stack now that mostly caught up, will have nightly science stacks to do. only 2/4x compute3 in stdscience manually for now.
• Serge looked at mysql on ipp0012 and ippdb02, restarted and slowly catching up.
• 15:20 MEH: odd, processing almost came to halt.. looks like stdscience pantasks is underloaded.. slowly coming back up. might as well do regular pantasks restart

Tuesday : 2013.03.19

mark is czar

• 00:30 MEH: datastore having intermittent proxy errors
• 07:05 MEH: nightly science still processing, turning chip.off to push the nightly warps out. looks like an OSS and a few other publishing jobs had fault and revert not automatic.
• 09:20 MEH: warps+diffs only running, something limiting processing (stsci06/07 over-targeted?) -- 3PI nightly now mostly finished, 3PI WSdiff a good number to do still
• 11:20 MEH: ippc17 is down?

• ipp020 still need a reboot in the morning once nightly science is finished

Wednesday : YYYY.MM.DD

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD