== PS1 IPP Czar Logs for the week 2011-05-23 - 2011-05-29 == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2011-05-23 ===
 * Lots of diffSkyfiles are failing because they need files that are only on ipp005. Set them to fault 0, quality 86
 * Found many STS raw images which had 3 instances. 1 on ipp005 (unavailable) 1 on one of the ippb nodes, and another on one of the ippb notes with zero size. These failed replications happened around 2011-04-30. Log of the fixes may be found [wiki:fixed_replications.2011-05-23 here] 
 * 11:00 EAM : ISP download was broken because one exposure (o5701i0164o02) was known to us, but removed from the summit list.  dsproductls returned inconsistent results because of this.  I have fixed this in a very lame way by subtracting 10 years from the date of the exposure (2011-05-20 09:30:50 to 2001-05-20 09:30:50).
 * 20:10 Bill : cleanup seemed bogged down. Lot's queued, little running. Restarted pantasks.

=== Tuesday : 2011-05-24 ===
 * turned off STS.2010.b label to allow nightlyscience to finish.
 * Queued chipRuns for obs_mode ESS and updated stdscience/input to turn this on in the future.

=== Wednesday : 2011.05.25 ===

=== Thursday : 2011.05.26 ===

=== Friday : 2011.05.27 ===

 * 12:25 EAM : rebooted ipp041 : kernel panic

=== Saturday : 2011.05.28 ===

=== Sunday : 2011.05.29 ===

 * 08:00 : ippdb01 had a disk error, but the replacement disk is missing, apparently.  we are degraded, but running.
 * 10:00 EAM : stopped and restarted all pantasks -- things seemed to be running slowly