== PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD == 
[[PageOutline]]

(Up to [wiki:PS1_IPP_CzarLogs PS1 IPP Czar Logs])

=== Monday : 2016.02.15 ===
 * 13:00 EAM : I am going to launch relphot using most ippx nodes and the ipps nodes, and probably some of the ipp080 - ipp104 nodes.  I'm restarting stdscience pantasks now to ensure nightly jobs are completed before launching relphot.  
 * 13:30 EAM : I have started relphot for the full PV3 master database.  this will likely take 24-48 hours.

=== Tuesday : YYYY.MM.DD ===

 * 10:30 CZW: ipp/replication pantasks running ippb03.0 OTA replication.
=== Wednesday : YYYY.MM.DD ===

=== Thursday : 2016.02.18 ===
 * 18:00 EAM : dvo operations are using most of the memory for the ipp0XX nodes.  I'm stopping the pantaskses and I will remove the storage nodes from the processing set for standard processing.
 * 21:20 EAM : summitcopy was stuck for a while and I had some trouble tracking down the reason.  apparently, two exposures had a failure in summitcopy advance.  using the command `pztool -toadvance -dbname gpc1 -limit 10` showed the problem exposures.  in summitcopy pantasks, the command `show.books` and then `show.book pzPendingAdvance` revealed the two exposures still on the books as if they were running -- they must have died in a way which pantasks dropped.  stopping summitcopy and calling `reset.advance` cleared that book and made things right again.

=== Friday : YYYY.MM.DD ===

=== Saturday : YYYY.MM.DD ===

=== Sunday : YYYY.MM.DD ===