== PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2011.10.24 ===
 * 08:30 large load spike on ippc18 ~08:20, what caused it? looks like only iqanalysis running. why is it running on ippc18?
 * 08:50 restarted distribution, set stare01-04 off manually for Cindy to upgrade disks. removing ippc10 from processing for Cindy to use parts in bringing ipp036 back up. also removed from pantasks_hosts.input in case servers restarted.
 * 10:00 adding "keep" as valid state in pxtools.c that pxUpdateRun can use, like "wait" but with more permanence.
 * 10:30 working on setting run states to keep/wait for faulted products due to ipp036 problems so not cleaned up.
 * 17:00 Cindy got ipp036 running again with 1 cpu, set neb-host to repair for now. working though stalled data products.
 * 22:00 mostly cleaned up, LAP is trickling through with nightly science. still some stuck in destreak holding it up though.
=== Tuesday : 2011-10-25 ===
 * 08:30 Corrupted chip file (1491546650.gpc1:ThreePi.nt:2011:10:25:o5859g0375o.414060:o5859g0375o.414060.ch.331016.XY63.ch.mk.fits) preventing camera stage correct execution. Reprocessing {{{perl runchipimfile.pl --chip_id 331016 --class_id XY63}}}
 * 09:00 ippdb02 is still ingesting the mysqldump (It's still ingesting the instance table).
=== Wednesday : YYYY.MM.DD ===

=== Thursday : YYYY.MM.DD ===

=== Friday : YYYY.MM.DD ===

=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===