PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Non-standard Processing

In an effort to improve communication, so we know what's going on better, HAF thinks we should list in the czar pages additional (non-standard) processing - so that we all know what's going on (no requirements for others to help run it). HAF will start:

HAF:

• ipp054 - ipp081: addstar processing for full force / diff. If you notice problems and need to reboot / stop this, please do the following:
  • contact HAF via email or phone (all should have my #)
  • if you feel addstar needs to be stopped and can't reach her:

           ssh ippdvo2@ippc19
           cd addstar.ipp054 (replace with machine that is problematic)
           pantasks_client
           stop
    

    • and then check status: wait for addstar run to finish, wait for minidvodb.premerge to finish (otherwise, risk corruption of mini)
• stsci03 is currently running ipptopsps for SAS39 FW. This is fairly resilient - no need to stop it if problem. you can access it via:

           ssh ippdvo2@ippc19
           screen -r fw
  

  • you can ctrl-c to stop it if necessary (but ipptopsps really doesn't care, it's fine.. it gracefully dies off if gpc1/nebulous die off, for example)

---

Monday : 2015.07.27

• HAF restarted summitcopy - shoved aside the summitExp faults (I think -- these are darks, and checkexp and others look "happy" now):

  delete from summitExp where fault = 110 and summit_id > 945000 and imfiles is null;;
  
  update summitExp set fault = 0 where fault = 110 and summit_id > 945000 ;
  

Tuesday : 2015.07.28

• 14:00 HAF : daily restart of pantasks

Wednesday : 2015.07.29

• 19:35 EAM : daily restart of pantasks

Thursday : 2015.07.30

• 04:40 Bill : reduced priority of ps_ud_MOPS label below nightly science labels since there seem to be a lot of requests pending
• 04:45 gpc1 progress is quite slow. gpc2 is using the cycles again.

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD