== PS1 IPP Czar Logs for the week 2013.09.09 - 2013.09.14 == [[PageOutline]] (Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs]) === Monday : 2013.09.09 === mark is czar * 08:40 MEH: finishing clearing broken LAP chips, then regular restart of stdsci * 09:05 MEH: 4 pantasks_servers running on ippc15 by ipp, should be only 3.. restarting most pantasks.. and 5 rather than 3 running on ippc04 -- culprits were publishing 2x, stack 2x, registration 2x loaded.. kind of excessive over-starting.. * wonder if the 2x registration has been source of odd registration hangup lately.. * 14:15 MEH: apparently ippc06 rebooted around @1345.. -- nothing on console.. someone bump it? * took down cleanup and detrend.. restarting.. * 14:30 MEH: ipp047 back online! apparently was never taken out of ippconfig/pantasks_hosts.input so taking out for now and putting into neb-host repair like siblings * 15:00 MEH: ippc06 rebooting itself again.. -- taking out of processing, restarting cleanup and detrend pantasks * 23:50 MEH: czarpoll fairly sluggish most of evening, not clear what from. going to restart === Tuesday : 2013.09.10 === * 07:10 MEH: ipp033 down, 238 exposures behind.. -- nothing on console, back up and registration+processing continues * ipp033 is in repair so unclear why would stall registration -- was a check_burntool problem for neb://ipp033.0, so used regtool to clear and not have to wait, but.. {{{ o6545g0361o XY52 0 check_burntool neb://ipp033.0/gpc1/20130910/o6545g0361o/o6545g0361o.ota52.fits #??? regtool -updateprocessedimfile -exp_id 654541 -class_id XY52 -set_state pending_burntool -dbname gpc1 }}} * 07:55 MEH: czar stuff is very slow, looks like ippdb01 is very busy still. dvo related? === Wednesday : YYYY.MM.DD === === Thursday : YYYY.MM.DD === === Friday : YYYY.MM.DD === === Saturday : YYYY.MM.DD === === Sunday : YYYY.MM.DD ===