IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links

Changes between Version 1 and Version 2 of PS1_IPP_Czarlog_20170605


Ignore:
Timestamp:
Jun 6, 2017, 10:16:43 PM (9 years ago)
Author:
Mark Huber
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • PS1_IPP_Czarlog_20170605

    v1 v2  
    77 * MEH: again fixing stalled stamps for QUB
    88 
    9 === Tuesday : YYYY.MM.DD ===
    10 
     9=== Tuesday : 2017.06.06 ===
     10 * MEH: Heather wondering issue w/ ippx012 being down -- ganglia shows offline early yesterday, console shows
     11{{{
     12<Jun/05 06:14 am>ippx012 login: [1679460.508553] ------------[ cut here ]------------
     13<Jun/05 06:14 am>[1679470.992940] kernel BUG at kernel/timer.c:926!
     14<Jun/05 06:14 am>[1679471.414703] invalid opcode: 0000 [#1] SMP
     15<Jun/05 06:15 am>[1679473.103464] Modules linked in: adm1021 ipv6 dm_mod joydev usbhid microcode igb ehci_hcd uhci_hcd i2c_i801 ioatdma coretemp i2c_core pcspkr usbcore acpi_cpufreq mperf dca usb_common sg processor button thermal_sys
     16<Jun/05 06:15 am>[1679506.734049] CPU 10
     17<Jun/05 06:15 am>[1679506.736226] Pid: 14368, comm: apache2 Not tainted 3.7.6 #1 Supermicro X8DTT/X8DTT
     18<Jun/05 06:15 am>[1679508.835724] RIP: 0010:[<ffffffff8103882d>]  [<ffffffff8103882d>] add_timer_on+0x34/0x78
     19<Jun/05 06:15 am>[1679510.520854] RSP: 0018:ffff88183f343e10  EFLAGS: 00010286
     20}}}
     21  * no remote power management yet, Haydn reminds would reboot 4 nodes ippx009-x012 and Heather would prefer to wait until next week to just individually reboot w/ physical button press
     22 * MEH: more fixing of error_cleaned data so pstamp and updates can work
     23 * MEH: ippc18/homedir was quickly running out of space -- Heather cleaned up logs to make space for ongoing, regular batch running, the warning cron (ipp@ippc18) has been enabled to alert when gets to minimal levels
     24 * MEH: K2.nightlyscience processing chip-cam-warp automated again and should be checked on by czars like the rest of nightly processing for next month or so again -- WSdiffs are still managed externally by me
    1125=== Wednesday : YYYY.MM.DD ===
    1226