PS1_IPP_Czarlog_20161031 – Pan-STARRS IPP

Context Navigation

Version 17 (modified by Mark Huber, 10 years ago) ( diff )
--

MEH: NCU/MOPS processing today, upper ippx and ipps nodes use continues
MEH: ippMonitor has had ippdb03 red for a while -- password for ippMonitor needed to be fixed in config -- showing slave status properly now
MEH: relastro reported done at meeting -- should reallocated nightly nodes back to default (x0,1->s3,4) since Heather using for next loading processed
- oddly seems s6 also commented out -- # EAM 2016.05.04 : remove s6 nodes to test load for ITS move -- is this still necessary? -- no, was a short time test and the 3x should have been put back into processing

MEH: NCU/MOPS processing mostly finished except for ipps node use (and normal nightly c2 node use) -- waiting for request for products as needed now
13:20 MEH: Haydn taking one of datanodes on 10G offline to test something with the cable? setting processing to stop -- finished and restarted all pantasks for nightly processing
- many of nightly datanodes are red... cleaning some of the NCU diffs to make space since appears not any other space to be freed

9:00 CZW: Processing stopped to allow for switchover to N5K switch for ippdb01.
12:45 CZW: Processing resumed.
16:25 CZW: Pantasks restart, even if we didn't have data last night.
MEH: QUB updating modest amount of NCU diffs for stamps off and on possibly (regularly part of ~ippqub/stdscience_ws and independent of nightly processing for MOPS)
MEH: SNIaF updates running on ipps nodes in ~ippmops/stdscience

MEH: reconfig for nightly QUB targeted processing -- finished until after midnight possibly tonight
16:30 CZW: Restart of pantasks after rebuild of ippTools. This picks up the change to prioritize the download of image files that do not have an exp_name LIKE 'c%' above those that do match that exp_name.
MEH: cleaning up remaining NCU WSdiff to hopefully free up more space for normal nightly data..

Note: See TracWiki for help on using the wiki.