== PS1 IPP Czar Logs for the week 2011.11.21 - 2011.11.27 == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2011.11.21  ===
 * 09:20 ippc12 removed from processing for Cindy to swap out parts with ipp029. ipp029 neb-host set to down. when finished ipp029 neb-host repair again and put into full stdscience and distribution processing test (LAP).
 * 10:10 ipp055 set off/on in processing to reset pcontrol for coredumpsize 0 (just removed from deepstack over weekend to avoid 13GB cores) 
 * 16:00 Sine waves are back again [[Image(ipp_before.png)]]
 * 16:05 ipp009 was overwhelmed by three undead pantasks_client started by czarpoll on 11/19. Roy killed them explicitly and the waves disappeared
[[Image(ipp009.jpg)]]
[[Image(ipp_after.jpg)]]
 * 11:59 Mark: stdscience running slow, restarting in 30 if still low. restart helped.

=== Tuesday : 2011.11.22 ===
Chris is czar
 * 11:30 Mark: ipp-20111110 rebuild with Gene's psphotStack fixes added. processing stopped and restarted.
=== Wednesday : YYYY.MM.DD ===
 * 03:10 (HST, 510 PDT) Mark: ipp066 seems to have gone down ~10pm. not in .Consolerc so manually added (cab8) and rebooted. added quick log at [wiki:ipp066_log]
=== Thursday : YYYY.MM.DD ===

=== Friday : YYYY.MM.DD ===

=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===