== PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2011.10.03 ===
 * 02:50 odd squelching behavior again on processing when plenty of exposures available. stdscience pantasks looks to go completely empty, then reloads. looks like started ~02:20 based on ganglia past hour load plot.
 * 03:10 Mark: restarted stdscience to see if helps. fully reloaded by ~03:50.
 * 04:10 looks to be happening again. no condor running so wasn't that. whatever it is also involves ippc11, looks like iqanalysis is running. processing back to constant levels ~04:40.
 * 04:50 back again, no iqanalysis on ippc11. ssh into other systems 2-5x longer.
 * 05:50 looks like processing moving back to regular levels? (by 06:05 yes) this behavior not clear in the ganglia 1d span, only hour span. seems to easily reduce the chip-warp rate to ~30/hr.
 * 12:40 Mark: off'd the chip.revert until the missing instance file script is tested to stop filling up the pantasks log.
 * 23:00 ipp029 unresponsive, ran neb-host down ipp029
 * 23:47 CZW: cycled power on ipp029 as it was preventing LAP destreak from finishing correctly, holding up further processing.
=== Tuesday : 2011.10.04 ===

 * 06:45: roy: everything down from summit (not much, only 21 science exposures)
=== Wednesday : YYYY.MM.DD ===

=== Thursday : YYYY.MM.DD ===

=== Friday : YYYY.MM.DD ===

=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===