PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : 2016.03.14

  • 04:38 MEH: registration backing up it looks like because exposure o7461g0601 stalled in summitcopy ~1500s, manually clearing..
  • 08:30 EAM: processing is done for the night. i'm restarting relastro clients
  • 13:55 MEH: stdsci and cleanup to stop while changing label to OSS.nightlyscience.mopshold for both sets of R13S2 WW diffims from OSS.20160314 (normally would be cleaned tomorrow afternoon) -- Serge will let us know when can clean up
  • 16:00 MEH: regular restart of pantasks now that cleanup is finished
  • 22:18 MEH: the current 6x group x2+x3 gives stdsci ~614 active nodes is probably excessive (was ~530? and well overpowered), particularly w/ the odd behavior the xnodes seems to have.

Tuesday : 2016.03.15

  • 07:10 MEH: clear OSS fault 4 warp (cannot build growth curve) problem
    warptool -dbname gpc1 -updateskyfile -set_quality 42 -skycell_id skycell.1115.041  -warp_id 1703288 -fault 0
    
  • using only 3x group x2+3 loaded instead of 6x and processing rate kept up to the normal ~60 exp/hr
  • 07:30 MEH: 73 WWdiff .multi type running, thought these were going to be turned off..

Wednesday : 2016-03-16

  • 13:30 MEH: sending OSS.nightlyscience.mopshold, OSS.20160314 to cleanup now that MOPS is finished with them
  • 15:45 CZW Restarting ipptest/replication to cull down over-replicated copies. This will then replicate copies that were missed in the previous iteration.
  • 15:55 CZW Restarting ipp/* pantasks for nightly reboot. I am also installing the regtool update that prevents the sunset/burntool issue from the weekend from popping up in the future.

Thursday : 2016.03.17

  • 01:25 MEH: (email notes to add) -- for o7464g0319o ota14
  • 01:42 MEH: watching another exposure stalled on check_burntool case o7464g0395o ota14, no error in registration log but been >10 min so far -- will manually clear if doesn't w/in 35 minutes -- didn't clear w/in 35 minutes and registration getting ~100 behind again... manually clearing..

Friday : 2016.03.18

  • 20:45 EAM : stopping pantasks for restart

Saturday : 2016.03.18

  • 14:30 EAM: stopping pantasks for restart

Sunday : YYYY.MM.DD