== PS1 IPP Czar Logs for the week 2014.09.01 - 2014.09.07 == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2014.09.01 ===
 * 14:30 EAM : stsci12 was down per Mark's message -- unresponsive on console.  I power-cycled it and it came OK.
 * 22:00 EAM : ipp036 was down, no message on console. power cycled it  


=== Tuesday : YYYY.MM.DD ===
 * 10:15 MEH: nightly rate low and net i/o <1G after Gene turned off local pv3 processing, so adding 2x c2 into stdsci (at least to recover systems turned off) -- 
 * 13:10 MEH: nightly not likely to finish before cleanup triggered @2pm, turning off cleanup pantasks until nightly finished so not to bog down the 5 data nodes further
 * 14:10 ipp040 being taken down for PS swap -- Gene set to stop and control reap
  * MEH: neb-host down
  * MEH: oddly unwant also 5, returned back to 20 when started up again  
  * MEH: w/ ipp040 down, finding missing flats on ippb02
{{{
/data/ippb02.1/nebulous/04/c5/1024845079.gpc1:flatcorr.20100124:GPC1.FLATTEST.305:GPC1.FLATTEST.305.XY15.co.fits
}}}
  * MEH: oddly, normal hosts off are back on in stdsci.. ipp032,060,064.. and newer off ones to beware of crashes.. 047,034,035,036.. including stsciXX -- so maybe better cmd to use instead of this after control reap all? -- need to do normal restart of stsci instead?
{{{
control run all
}}}
=== Wednesday : YYYY.MM.DD ===

=== Thursday : YYYY.MM.DD ===

=== Friday : YYYY.MM.DD ===

=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===