PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : 2018.03.19

• MEH: had normal nightly data nodes .0 ipp100,101,102,103,104,118,119,122 in repair in the past due to lack of space, plenty of space for a few weeks now but not put back up... putting neb-host up to help keep them balanced with .1 partitions...

Tuesday : 2018.03.24

• MEH: pstamp parsing can get behind w/ large QUB requests, bump parse nodes from 20->50 of 60 available (~ippitc/pstamp/input set.parse.limit 50) leaving a few to do stamps and push the requests through more quickly (estimate ~20min now rather than ~hour typically) until some sort or priority can be added to the parsing for MOPS
• MEH: ps_ud_% cleanup moved from 1200-1400 HST to 1800 HST to help avoid update cleanups blocking stamps completing for QUB (~ippitc/pstamp/input adding shift.trange.update.cleanup)

Wednesday : 2018.03.21

• MEH: Haydn shutting down nodes at ATRC @0700 HST -- setting neb-host down on ippb00-03,06-15
  • ippb00 not powering up, Haydn needs to look into tomorrow
  • ippb01-03,06 back to neb-host repair; ippb07-15 neb-host up like before powered down
• MEH: rebuild ipp-20170121 ippconfig for retaining warps for 7 days, diffs for 3 days now (likely will be set to keep warps even longer, ipp-20170121 nightly_science.config) and adding misc updates for K2 and config for Bright nightly processing -- test quad run and checked by Serge as required for any ops tag changes

Thursday : YYYY.MM.DD

Friday : 2018.03.23

• MEH: misc faults on ippmonitor for ps_ud_MOPS -- a mess of missing LAP files from the earlier shuffle with stsci nodes still not restored or dealt with

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD