PS1 IPP Czar Logs for the week YYYY.MM.DD - YYYY.MM.DD

(Up to PS1 IPP Czar Logs)

Monday : YYYY.MM.DD

  • 16:00 HAF restarting pantasks in preparation for tonight

Tuesday : 2016.02.02

  • 14:14 MEH: changing retention time for nightly warp+diffs for QUB from 3->2 days starting tomorrow until disk space issue can be improved
  • 20:45 MEH: stdscience was down, dmesg had segfault -- restarted fine it seems
    [11257288.997177] pantasks_server[27837]: segfault at 4f76e48 ip 0000000000408a4e sp 00000000425acf20 error 4 in pantasks_server[400000+16000]
    

Wednesday : YYYY.MM.DD

Thursday : 2016-02-04

  • 16:00 CZW: Starting the rsync of data from ipp033-ipp053 to ipp100-ipp104. Running in screen as ipp on stare03, in directory /data/ippc19.0/home/ipp/replication/ipp10X. This is based on the stsci shuffle method.
  • 17:35 EAM: stopping and restarting pantaskses
  • 22:40 EAM: ippdb03 is offline due to a bad system disk. since ippdb03 is offline (bad system disk), ippMonitor is down. i have install the web pages on ippc18 instead, so point your browser at:
    ippc18.ifa.hawaii.edu:/ippMonitor
    

instead for now.

Friday : 2016.02.05

  • 06:45 EAM: ipp017 crashed, rebooted it
  • 16:00 CZW: Gavin has set up automount for the new storage nodes, with the exception of ipp029, ippx052, and ippdb03, which were down at the time.
  • 16:19 CZW: Which should have been a warning that something was up with ipp029. Logged onto console, looks like a kernel panic. Power cycled.

Saturday : 2016.02.06

  • 20:30 EAM: restarting pantasks

Sunday : YYYY.MM.DD