== PS1 IPP Czar Logs for the week 2011.02.06 - 2011.02.10 == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2011.02.06 ===
  
Roy is czar

 * 09:00: only 70 science images, all through the pipeline
 * 12:45 Serge: replication/shuffling seemed to be stuck. Following Chris advice, I shut it down and restarted it.
 * 13:30: Roy: ran Bill's script to repair 'lost' instance of Exp: 193145, OTA: 15

{{{
repair_bad_instance -e 193145 -c XY15 -l
}}}

 * 13:40 fixing bad burn.tbl as example for Roy

{{{
neb-stat --validate neb://ipp049.0/gpc1/20100629/o5376g0254o/o5376g0254o.ota54.burn.tbl
ipp_apply_burntool_single.pl --class_id XY54 --exp_id 188719 --dbname gpc1 --this_uri neb://ipp049.0/gpc1/20100629/o5376g0254o/o5376g0254o.ota54.fits --previous_uri neb://ipp049.0/gpc1/20100629/o5376g0253o/o5376g0253o.ota54.fits
neb-repair neb://ipp049.0/gpc1/20100629/o5376g0254o/o5376g0254o.ota54.burn.tbl
}}}

 * 13:50: Roy: Trying one on my own:

{{{
neb-stat -validate neb://ipp051.0/gpc1/20100719/o5396g0026o/o5396g0026o.ota64.burn.tbl
ipp_apply_burntool_single.pl --class_id XY64 --exp_id 193152 --dbname gpc1 --this_uri neb://ipp051.0/gpc1/20100719/o5396g0026o/o5396g0026o.ota64.fits --previous_uri neb://ipp051.0/gpc1/20100719/o5396g0025o/o5396g0025o.ota64.fits
neb-repair neb://ipp051.0/gpc1/20100719/o5396g0026o/o5396g0026o.ota64.burn.tbl
}}}

 * same for:

{{{
--class_id XY65 --exp_id 193166
--class_id XY51 --exp_id 193145
}}}

 * and finally:

{{{
repair_bad_instance -e 193162 -c XY26 -r
}}}

 
=== Tuesday : 2011.02.07 ===

Roy is czar

 * 08:00: Roy: No data last night
 * 08:00: Roy: LAP stuck again. One chip with a single corrupted copy and two with no copies at all, so:


{{{
repair_bad_instance -e 183688 -c XY26 -l
repair_bad_instance -e 183688 -c XY32 -l
repair_bad_instance -e 183664 -c XY65 -l
}}}

 * 09:42 postage stamp server and data store shut down for database recovery.
 * 10:00: Roy: stuck stacks, so:

{{{
stacktool -revertsumskyfile -fault 2 -label LAP.ThreePi.20110809 -dbname gpc1
}}}
 * 11:18 A couple of distribution fileset faults were lingering probably due to ippc17 failures. The failure was dsreg "fileset already exists" deleted them and the registrations completed.
 * 11:40: Roy: more stuck stacks:

{{{
stacktool -revertsumskyfile -fault 2 -label LAP.ThreePi.20110809 -dbname gpc1
}}}

=== Wednesday : YYYY.MM.DD ===

=== Thursday : YYYY.MM.DD ===

=== Friday : YYYY.MM.DD ===

=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===