PS1 IPP Czar Logs for the week 2016.11.07 - 2016.11.13

(Up to PS1 IPP Czar Logs)

Monday : 2016.11.07

* 9:15am requeued 3-4 for serge difftool -dbname gpc1 -definewarpwarp -exp_id 1162564 -template_exp_id 1162581 -backwards -set_workdir neb://@HOST@.0/gpc1/OSS.nt/2016/11/07 -set_dist_group SweetSpot -set_label OSS.nightlyscience -set_data_group OSS.20161107 -set_reduction SWEETSPOT -simple -rerun

* 15:30 CZW: Restarting shuffle tests. Running with 20 clients on the stare nodes (server host stare02), with the pantasks root in /data/ippc19.0/home/watersc1/itc_sync_script.20161026/test_pant. If this looks like it's causing problems, please stop but not shutdown the pantasks (so I can grab some statistics). The main operations are reading files and calculating md5sums, copying to ITC, and doing more md5sums, so the impact should be minimal.

Tuesday : 2016.11.08

  • MEH: bumping up pstamp while QUB does a large request of updates of recent data cleaned to cope with low storage space

Wednesday : 2016.11.09

  • 18:45 EAM: restarting pantasks

Thursday : 2016.11.10

Friday : 2016.11.11

  • MEH: bumping up pstamp while QUB does a large request

Saturday : 2016.11.12

Sunday : 2016.11.13

  • 03:30 MEH: ipp060 had excessive dvo job running and looks like had kernel panic ~00:46 -- power cycle to try and bring nightly processing back on...
  • 11:05 EAM: some failures on ipp060 got left in the pantasks book. I restarted pantasks and they are running again.