== PS1 IPP Czar Logs for the week 2013.05.27 - 2013.06.02 == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2013.05.27 ===
 * 02:30 MEH: ipp033 crashed/unresponsive ~4hrs ago.. holding up nightly download/processing.. rebooting
 * 03:00 MEH: system still wedged.. restarting summitcopy and registration to clear queues and logs to watch
 * 03:20 MEH: still wedged.. nfs trouble on ipp033 still, restarting and looks to be clearing
 * 08:50 MEH: stdscience struggling.. doing a regular restart to finish up nightly science
 * 13:00 MEH: odd SMF full but MIA stalling warp -- manually remaking
{{{
neb entry neb://any/gpc1/ThreePi.nt/2013/05/27//o6439g0376o.615201/o6439g0376o.615201.cm.816310.smf not found

perl ~ipp/src/ipp-20130307/tools/runcameraexp.pl --redirect-output --cam_id 816310
}}}
=== Tuesday : 2013.05.28 ===
Bill is czar today
 * 12:08 queued STS exposures from 2009 and 2010 to be processed continuing to use label STS.rp.20130508
  * not running deep stacks so might as well add compute3 back into stdscience 
=== Wednesday : 2013.05.29 ===

=== Thursday : 2013.05.30 ===

=== Friday : 2013.05.31 ===

=== Saturday : 2013.06.01 ===

=== Sunday : 2013.06.02 ===