PS1 IPP Czar Logs for the week 2016.10.31 - 2016.11.06

(Up to PS1 IPP Czar Logs)

Monday : 2016.10.31

• MEH: NCU/MOPS processing today, upper ippx and ipps nodes use continues
• MEH: ippMonitor has had ippdb03 red for a while -- password for ippMonitor needed to be fixed in config -- showing slave status properly now
• MEH: relastro reported done at meeting -- should reallocated nightly nodes back to default (x0,1->s3,4) since Heather using for next loading processed
  • oddly seems s6 also commented out -- # EAM 2016.05.04 : remove s6 nodes to test load for ITS move -- is this still necessary? -- no, was a short time test and the 3x should have been put back into processing

Tuesday : 2016.11.01

• MEH: NCU/MOPS processing mostly finished except for ipps node use (and normal nightly c2 node use) -- waiting for request for products as needed now
• 13:20 MEH: Haydn taking one of datanodes on 10G offline to test something with the cable? setting processing to stop -- finished and restarted all pantasks for nightly processing
  • many of nightly datanodes are red... cleaning some of the NCU diffs to make space since appears not any other space to be freed

Wednesday : 2016.11.02

• 9:00 CZW: Processing stopped to allow for switchover to N5K switch for ippdb01.
• 12:45 CZW: Processing resumed.
• 16:25 CZW: Pantasks restart, even if we didn't have data last night.
• MEH: QUB updating modest amount of NCU diffs for stamps off and on possibly (regularly part of ~ippqub/stdscience_ws and independent of nightly processing for MOPS)
• MEH: SNIaF updates running on ipps nodes in ~ippmops/stdscience

Thursday : 2016.11.03

• MEH: reconfig for nightly QUB targeted processing -- finished

Friday : 2016.11.04

Saturday : 2016.11.05

Sunday : 2016.11.06