== PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD == [[PageOutline]] (Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs]) === Monday : 2017.03.06 === * MEH: ipp001 disk space low, running mysql cleanup scripts for gpc1 dumps * MEH: ipp078 in repair again so Haydn can work on it this week -- did today, swapped SATA cables and backup okay -- leave neb-host repair w/o nightly load, just Heather may have jobs again on it * MEH: Gene fix for dvoImagesAtCoords -- rebuild ipp ipp-20170121/Ohana onward and pstamp seems to produce chips okay again === Tuesday : YYYY.MM.DD === === Wednesday : YYYY.MM.DD === === Thursday : 2017-03-09 === * 17:20 CZW: Set ITC nodes to xattr=0 in nebulous. This field controls replication/new copy status, and will now allow first copies to be generated on the ITC nodes (ipp105-117). * 18:40 CZW: Stopped apache on ippc17 and ippc30 to ensure no problems with the ITC switchover test tonight. === Friday : 2017-03-10 === * 13:30 CZW: Restarted ipptest/replication with 50 jobs to prevent overloading of MRTCB-ITC connection. * MEH: running ippqub WSdiffs and updates during ippitc processing of MOPS test chunk * MEH: all nodes ipp054-ipp104 into repair to prevent scattered data going there in preparation to move to ITC -- all mrtc-b nightly to target ipp118-122 * MEH: distribution uses targeting parameters in gpc1:rcDestination:dbhost for ippc17, will take a little time to setup again for ipp113 and distribution is off for now * MEH: pstamp@mrtc-b (just to keep all active processing there) seems mostly functional -- * MEH: restarting summitcopy, registration, stdscience with updated nodes (replace ippx001-12 and ipp090,91,92 shutdown) and for targeting ipp118-122 only -- running @mrtc-b, all individually started w/ start_server.sh * MEH: cleanup wasn't on/cleared before ipp090-092 were turned off and packed -- may be issues -- leaving off unless need to clear issues === Saturday : YYYY.MM.DD === === Sunday : YYYY.MM.DD ===