PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : 2016.02.15

• 13:00 EAM : I am going to launch relphot using most ippx nodes and the ipps nodes, and probably some of the ipp080 - ipp104 nodes. I'm restarting stdscience pantasks now to ensure nightly jobs are completed before launching relphot.
• 13:30 EAM : I have started relphot for the full PV3 master database. this will likely take 24-48 hours.

Tuesday : YYYY.MM.DD

• 10:30 CZW: ipp/replication pantasks running ippb03.0 OTA replication.

Wednesday : YYYY.MM.DD

Thursday : 2016.02.18

• 18:00 EAM : dvo operations are using most of the memory for the ipp0XX nodes. I'm stopping the pantaskses and I will remove the storage nodes from the processing set for standard processing.
• 21:20 EAM : summitcopy was stuck for a while and I had some trouble tracking down the reason. apparently, two exposures had a failure in summitcopy advance. using the command pztool -toadvance -dbname gpc1 -limit 10 showed the problem exposures. in summitcopy pantasks, the command show.books and then show.book pzPendingAdvance revealed the two exposures still on the books as if they were running -- they must have died in a way which pantasks dropped. stopping summitcopy and calling reset.advance cleared that book and made things right again.

Friday : 2016.02.19

• 09:40 MEH: clearing unrecoverable fault 4 (warp curve of growth) and fault 5 (diff)

  warptool -dbname gpc1 -updateskyfile -set_quality 42 -skycell_id skycell.0534.058 -warp_id 1688791 -fault 0
  difftool -dbname gpc1 -updatediffskyfile -set_quality 42 -skycell_id skycell.1550.056 -diff_id 1341246  -fault 0
  

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD