for >1hr as load continued to climb (was unable to login or connect to disks MEH). reboot ok

• Checking root filesystem .../dev/sda3 has gone 264 days without being checked, check forced. /dev/sda3: 317637/2003120 files (0.1% non-contiguous), 1859567/8000000 blocks

on console

fff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:00 pm>[1974761.848939]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Jul/28 12:00 pm>[1974761.855916]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb
<Jul/28 12:00 pm>[1974761.861320]  [<ffffffff810546e2>] ? kthread+0x7d/0x85
<Jul/28 12:00 pm>[1974761.866855]  [<ffffffff81003794>] ? kernel_thread_helper+0x4/0x10
<Jul/28 12:00 pm>[1974761.873471]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Jul/28 12:00 pm>[1974761.878929]  [<ffffffff81003790>] ? kernel_thread_helper+0<Jul/28 12:00 pm>
                 <Jul/28 12:00 pm>[1974761.885546] Code: d0 48 23 02 48 85 c0 49 89 87 40 6d c1 81 74 20 48 8b 05 60 b8 af 00 41 8d b6 f0 00 00 00 48 89 df ff 90 e0 00 00 00 eb 02 f3 90 <83> 3b 00 75 f9 48 8b 7d d0 49 c7 87 00 6d c1 81 00 00 00 00 49 
<Jul/28 12:00 pm>[1974761.906895] Call Trace:
<Jul/28 12:00 pm>[1974761.909772]  [<ffffffff810281b2>] ? flush_tlb_page+0x6e/0x76
<Jul/28 12:00 pm>[1974761.915939]  [<ffffffff810cc352>] ? page_check_address+0xb0/0x106
<Jul/28 12:00 pm>[1974761.922545]  [<ffffffff81027599>] ? ptep_clear_flush_young+0x20/0x26
<Jul/28 12:00 pm>[1974761.929426]  [<ffffffff810cc62f>] ? page_referenced_one+0x65/0xe5
<Jul/28 12:00 pm>[1974761.935993]  [<ffffffff812bba1d>] ? prio_tree_next+0x208/0x22b
<Jul/28 12:00 pm>[1974761.942322]  [<ffffffff810cd6f1>] ? page_referenced+0x2e0/0x335
<Jul/28 12:00 pm>[1974761.948736]  [<ffffffff810e5fb2>] ? mem_cgroup_del_lru_list+0x7b/0x8e
<Jul/28 12:00 pm>[1974761.955654]  [<ffffffff810b6e86>] ? isolate_pages_global+0x1c6/0x23f
<Jul/28 12:00 pm>[1974761.962500]  [<ffffffff810b7ab2>] ? shrink_active_list+0x1<Jul/28 12:00 pm>
                 <Jul/28 12:00 pm>[1974761.969183]  [<ffffffff810b8003>] ? shrink_zone+0x484/0xb86
<Jul/28 12:00 pm>[1974761.975258]  [<ffffffff812734b7>] ? xfs_perag_get+0x29/0x99
<Jul/28 12:00 pm>[1974761.981352]  [<ffffffff81058774>] ? up_read+0x1e/0x36

<Jul/28 12:00 pm>[1974761.986916]  [<ffffffff810b931a>] ? kswapd+0x61a/0x8bb
<Jul/28 12:00 pm>[1974761.992519]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:00 pm>[1974761.999186]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Jul/28 12:00 pm>[1974762.006153]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb
<Jul/28 12:00 pm>[1974762.011603]  [<ffffffff810546e2>] ? kthread+0x7d/0x85
                 <Jul/28 12:00 pm>[1974762.017085]  [<ffffffff81003794>] ? kernel_thread_helper+0x4/0x10
<Jul/28 12:00 pm>[1974762.023702]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Jul/28 12:00 pm>[1974762.029135]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Jul/28 12:00 pm>[1974803.417084] BUG: soft lockup - CPU#1 stuck for 61s! [sshd:9057]
<Jul/28 12:00 pm>[1974803.423524] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:00 pm>[1974803.457702] CPU 1 
<Jul/28 12:00 pm>[1974803.459772] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:00 pm>[1974803.498881] 
<Jul/28 12:00 pm>[1974803.500790] Pid: 9057, comm: sshd Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:00 pm>[1974803.508702] RIP: 0010:[<ffffffff8106a4e0>]  [<ffffffff8106a4e0>] smp_call_function_many+0x18c/0x1a5
<Jul/28 12:00 pm>[1974803.518532] RSP: 0018:ffff880c18747b20  EFLAGS: 00000202
<Jul/28 12:00 pm>[1974803.524316] RAX: 0000000000000020 RBX: ffff880c18747b60 RCX: 0000000000000020
<Jul/28 12:00 pm>[1974803.532100] RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:00 pm>[1974803.539908] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81b24210
<Jul/28 12:00 pm>[1974803.547783] R10: ffffffff812734b7 R11: 0000000000000286 R12: ffff880c18747ae0
<Jul/28 12:00 pm>[1974803.555589] R13: 0000000000000046 R14: ffff880c18747ae0 R15: 0000000000000000
<Jul/28 12:00 pm>[1974803.563437] FS:  00007f23edbf76f0(0000) GS:ffff880002620000(0000) knlGS:0000000000000000

<Jul/28 12:00 pm>[1974803.572291] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:00 pm>[1974803.578509] CR2: 00007fffec51b310 CR3: 0000000c239bf000 CR4: 00000000000006e0
<Jul/28 12:00 pm>[1974803.586327] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:00 pm>[1974803.594123] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:00 pm>[1974803.601959] Process sshd (pid: 9057, threadinfo ffff880c18746000, task ffff880c187a0000)
<Jul/28 12:00 pm>[1974803.610778] Stack:
<Jul/28 12:00 pm>[1974803.613202]  01ff880c18747ba0 0000000000000000 ffffffff810b086c 0000000000000040
<Jul/28 12:00 pm>[1974803.621033] <0> 0000000000000000 ffffffff810b086c 0000000000000003 0000000000000002
<Jul/28 12:00 pm>[1974803.629469] <0> ffff880c18747b70 ffffffff8106a519 ffff880c18747ba0 ffffffff81040d28
<Jul/28 12:00 pm>[1974803.638290] Call Trace:
<Jul/28 12:00 pm>[1974803.641166]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:00 pm>[1974803.647493]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:00 pm>[1974803.653840]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:00 pm>[1974803.660279]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:00 pm>[1974803.666173]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:00 pm>[1974803.672848]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:00 pm>[1974803.679886]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:00 pm>[1974803.685407]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:00 pm>[1974803.692013]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:00 pm>[1974803.697765]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:00 pm>[1974803.703941]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:00 pm>[1974803.709999]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc<Jul/28 12:00 pm>

<Jul/28 12:00 pm>[1974803.716093]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:00 pm>[1974803.722414]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:00 pm>[1974803.728498]  [<ffffffff81063516>] ? lockdep_init_map+0x9f/0x52e
<Jul/28 12:00 pm>[1974803.734967]  [<ffffffff812c4f51>] ? __raw_spin_lock_init+0x31/0x54
<Jul/28 12:00 pm>[1974803.741721]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:00 pm>[1974803.747470]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:00 pm>[1974803.753812]  [<ffffffff810e8850>] ? fd_install+0x30/0x5f
<Jul/28 12:00 pm>[1974803.759592]  [<ffffffff810e8876>] ? fd_install+0x56/0x5f
<Jul/28 12:00 pm>[1974803.765426]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:00 pm>[1974803.771122]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:00 pm>[1974803.776897]  [<ffffffff810029ab>] ? system_call_fastpath+0<Jul/28 12:00 pm>
                 <Jul/28 12:00 pm>[1974803.783571] Code: 48 c7 c7 d0 21 a0 81 4c 89 60 08 4c 89 25 f9 7c 99 00 e8 59 51 4b 00 0f ae f0 48 89 df ff 15 f0 d1 a2 00 80 7d c7 00 75 04 eb 0a <f3> 90 41 f6 44 24 20 01 75 f6 48 83 c4 18 5b 41 5c 41 5d 41 5e 
<Jul/28 12:00 pm>[1974803.804920] Call Trace:
<Jul/28 12:00 pm>[1974803.807774]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:00 pm>[1974803.814131]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:00 pm>[1974803.820502]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:00 pm>[1974803.826934]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:00 pm>[1974803.832853]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:00 pm>[1974803.839512]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:00 pm>[1974803.846515]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:00 pm>[1974803.852016]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:00 pm>[1974803.858634]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b

<Jul/28 12:00 pm>[1974803.864361]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:00 pm>[1974803.870509]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:00 pm>[1974803.876593]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:00 pm>[1974803.882625]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:00 pm>[1974803.888949]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:00 pm>[1974803.895033]  [<ffffffff81063516>] ? lockdep_init_map+0x9f/0x52e
<Jul/28 12:00 pm>[1974803.901450]  [<ffffffff812c4f51>] ? __raw_spin_lock_init+0x31/0x54
<Jul/28 12:00 pm>[1974803.908155]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:00 pm>[1974803.913845]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:00 pm>[1974803.920173]  [<ffffffff810e8850>] ? fd_install+0x30/0x5f
<Jul/28 12:00 pm>[1974803.925986]  [<ffffffff810e8876>] ? fd_install+0x56/0x5f
<Jul/28 12:00 pm>[1974803.931789]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:00 pm>[1974803.937494]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:00 pm>[1974803.943260]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:01 pm>[1974816.301255] BUG: soft lockup - CPU#3 stuck for 61s! [tcsh:3708]
<Jul/28 12:01 pm>[1974816.307680] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:01 pm>[1974816.341822] CPU 3 
<Jul/28 12:01 pm>[1974816.343902] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:01 pm>[1974816.378389] 
<Jul/28 12:01 pm>[1974816.380252] Pid: 3708, comm: tcsh Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:01 pm>[1974816.388191] RIP: 0010:[<ffffffff8106a4e8>]  [<ffffffff8106a4e8>] smp_call_function_many+0x194/0x1a5
<Jul/28 12:01 pm>[1974816.398044] RSP: 0018:ffff880347d45b20  EFLAGS: 00000202

<Jul/28 12:01 pm>[1974816.403856] RAX: 0000000000000020 RBX: ffff880347d45b60 RCX: 0000000000000020
<Jul/28 12:01 pm>[1974816.411676] RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:01 pm>[1974816.419544] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81b24210
<Jul/28 12:01 pm>[1974816.427384] R10: ffffffff812734b7 R11: 0000000000000286 R12: ffff880347d45ae0
<Jul/28 12:01 pm>[1974816.435224] R13: 0000000000000046 R14: ffff880347d45ae0 R15: 0000000000000000
<Jul/28 12:01 pm>[1974816.443145] FS:  00007ff01d3fc6f0(0000) GS:ffff880002660000(0000) knlGS:0000000000000000
<Jul/28 12:01 pm>[1974816.452031] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:01 pm>[1974816.458274] CR2: 000000000065bbf8 CR3: 00000005ea477000 CR4: 00000000000006e0
<Jul/28 12:01 pm>[1974816.466094] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:01 pm>[1974816.473949] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:01 pm>[1974816.481793] Process tcsh (pid: 3708, threadinfo ffff880347d44000, task ffff8800b3493090)
<Jul/28 12:01 pm>[1974816.490617] Stack:
<Jul/28 12:01 pm>[1974816.493029]  01ff880347d45ba0 0000000000000000 ffffffff810b086c 0000000000000040
<Jul/28 12:01 pm>[1974816.500856] <0> 0000000000000000 ffffffff810b086c 0000000000000003 0000000000000002
<Jul/28 12:01 pm>[1974816.509328] <0> ffff880347d45b70 ffffffff8106a519 ffff880347d45ba0 ffffffff81040d28
<Jul/28 12:01 pm>[1974816.518150] Call Trace:
<Jul/28 12:01 pm>[1974816.521024]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:01 pm>[1974816.527342]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:01 pm>[1974816.533698]  [<ffffffff8106a519>] ? smp_call_function+0x20<Jul/28 12:01 pm>
                 <Jul/28 12:01 pm>[1974816.540113]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:01 pm>[1974816.545979]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:01 pm>[1974816.552656]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671

<Jul/28 12:01 pm>[1974816.559678]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:01 pm>[1974816.566262]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:01 pm>[1974816.571963]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:01 pm>[1974816.578149]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:01 pm>[1974816.584470]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:01 pm>[1974816.590516]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:01 pm>[1974816.596568]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:01 pm>[1974816.602896]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:01 pm>[1974816.608938]  [<ffffffff810c5179>] ? handle_mm_fault+0x6e3/0x73e
<Jul/28 12:01 pm>[1974816.615322]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:01 pm>[1974816.620890]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:01 pm>[1974816.626597]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:01 pm>[1974816.632952]  [<ffffffff810f6798>] ? set_close_on_exec+0x30<Jul/28 12:01 pm>
                 <Jul/28 12:01 pm>[1974816.639391]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Jul/28 12:01 pm>[1974816.646267]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:01 pm>[1974816.651960]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:01 pm>[1974816.657735]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:01 pm>[1974816.664417] Code: 89 60 08 4c 89 25 f9 7c 99 00 e8 59 51 4b 00 0f ae f0 48 89 df ff 15 f0 d1 a2 00 80 7d c7 00 75 04 eb 0a f3 90 41 f6 44 24 20 01 <75> f6 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f c9 c3 48 89 f8 55 
<Jul/28 12:01 pm>[1974816.685731] Call Trace:
<Jul/28 12:01 pm>[1974816.688604]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:01 pm>[1974816.694915]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:01 pm>[1974816.701253]  [<ffffffff8106a519>] ? smp_call_function+0x20<Jul/28 12:01 pm>

<Jul/28 12:01 pm>[1974816.707677]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:01 pm>[1974816.713595]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:01 pm>[1974816.720276]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:01 pm>[1974816.727326]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:01 pm>[1974816.733911]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:01 pm>[1974816.739629]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:01 pm>[1974816.745789]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:01 pm>[1974816.752126]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:01 pm>[1974816.758210]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:01 pm>[1974816.764268]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:01 pm>[1974816.770588]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:01 pm>[1974816.776683]  [<ffffffff810c5179>] ? handle_mm_fault+0x6e3/0x73e
<Jul/28 12:01 pm>[1974816.783090]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:01 pm>[1974816.793139]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:01 pm>[1974816.798879]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:01 pm>[1974816.805193]  [<ffffffff810f6798>] ? set_close_on_exec+0x30/0x85
<Jul/28 12:01 pm>[1974816.811596]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Jul/28 12:01 pm>[1974816.818484]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:01 pm>[1974816.824156]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:01 pm>[1974816.829965]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:01 pm>[1974817.375340] BUG: soft lockup - CPU#5 stuck for 61s! [sshd:25093]
<Jul/28 12:01 pm>[1974817.381844] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis

<Jul/28 12:01 pm>[1974817.415907] CPU 5 
<Jul/28 12:01 pm>[1974817.417977] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:01 pm>[1974817.452316] 
<Jul/28 12:01 pm>[1974817.454198] Pid: 25093, comm: sshd Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:01 pm>[1974817.462186] RIP: 0010:[<ffffffff8106a4e2>]  [<ffffffff8106a4e2>] smp_call_function_many+0x18e/0x1a5
<Jul/28 12:01 pm>[1974817.471982] RSP: 0018:ffff88035da9d978  EFLAGS: 00000202
<Jul/28 12:01 pm>[1974817.477757] RAX: 0000000000000020 RBX: ffff88035da9d9b8 RCX: 0000000000000020
<Jul/28 12:01 pm>[1974817.485569] RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:01 pm>[1974817.493348] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81b24210
<Jul/28 12:01 pm>[1974817.501154] R10: ffffffff812734b7 R11: 0000000000000286 R12: 0000000000000046
<Jul/28 12:01 pm>[1974817.508959] R13: ffff88035da9d938 R14: 0000000000000000 R15: 0000000000000046
<Jul/28 12:01 pm>[1974817.516784] FS:  00007f71cc38c6f0(0000) GS:ffff8800026a0000(0000) knlGS:0000000000000000
<Jul/28 12:01 pm>[1974817.525612] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:01 pm>[1974817.531848] CR2: 0000000000cc8058 CR3: 00000002d5944000 CR4: 00000000000006e0
<Jul/28 12:01 pm>[1974817.539654] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:01 pm>[1974817.547473] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:01 pm>[1974817.555314] Process sshd (pid: 25093, threadinfo ffff88035da9c000, task ffff880335739030)
<Jul/28 12:01 pm>[1974817.564229] Stack:
<Jul/28 12:01 pm>[1974817.566636]  01ff88035da9d9f8 0000000000000000 ffffffff810b086c 0000000000000040
<Jul/28 12:01 pm>[1974817.574440] <0> 0000000000000000 ffffffff810b086c 0000000000000003 0000000000000002
<Jul/28 12:01 pm>[1974817.582894] <0> ffff88035da9d9c8 ffffffff8106a519 ffff88035da9d9f8 ffffffff81040d28
<Jul/28 12:01 pm>[1974817.591759] Call Trace:

<Jul/28 12:01 pm>[1974817.594607]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:01 pm>[1974817.600926]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:01 pm>[1974817.607246]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:01 pm>[1974817.613635]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:01 pm>[1974817.619521]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:01 pm>[1974817.626185]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:01 pm>[1974817.633210]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:01 pm>[1974817.639810]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:01 pm>[1974817.645477]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:01 pm>[1974817.651625]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:01 pm>[1974817.658213]  [<ffffffff810e06b3>] ? __kmalloc_node_track_caller+0x9f/0x114
<Jul/28 12:01 pm>[1974817.665599]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:01 pm>[1974817.672206]  [<ffffffff81450996>] ? __alloc_skb+0x6f/0x168
<Jul/28 12:01 pm>[1974817.678180]  [<ffffffff8148f924>] ? tcp_current_mss+0x46/0x65
<Jul/28 12:01 pm>[1974817.684405]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:01 pm>[1974817.690994]  [<ffffffff8148677b>] ? tcp_sendmsg+0x1b2/0xa39
<Jul/28 12:01 pm>[1974817.697027]  [<ffffffff814471df>] ? sock_aio_write+0xdf/0xef
<Jul/28 12:01 pm>[1974817.703139]  [<ffffffff810ea282>] ? do_sync_write+0xc7/0x1<Jul/28 12:01 pm>
                 <Jul/28 12:01 pm>[1974817.709312]  [<ffffffff8104a535>] ? sigprocmask+0x30/0xc8
<Jul/28 12:01 pm>[1974817.715160]  [<ffffffff812937db>] ? security_file_permission+0x11/0x13
<Jul/28 12:01 pm>[1974817.722193]  [<ffffffff810eabb4>] ? vfs_write+0xc1/0x14a
<Jul/28 12:01 pm>[1974817.728007]  [<ffffffff810ead01>] ? sys_write+0x47/0x6e

<Jul/28 12:01 pm>[1974817.733688]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:01 pm>[1974817.740344] Code: c7 d0 21 a0 81 4c 89 60 08 4c 89 25 f9 7c 99 00 e8 59 51 4b 00 0f ae f0 48 89 df ff 15 f0 d1 a2 00 80 7d c7 00 75 04 eb 0a f3 90 <41> f6 44 24 20 01 75 f6 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f 
<Jul/28 12:01 pm>[1974817.761639] Call Trace:
<Jul/28 12:01 pm>[1974817.764520]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:01 pm>[1974817.770857]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:01 pm>[1974817.777178]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:01 pm>[1974817.783575]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:01 pm>[1974817.789460]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:01 pm>[1974817.796111]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:01 pm>[1974817.803156]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:01 pm>[1974817.809768]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:01 pm>[1974817.815491]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:01 pm>[1974817.821604]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:01 pm>[1974817.828198]  [<ffffffff810e06b3>] ? __kmalloc_node_track_caller+0x9f/0x114
<Jul/28 12:01 pm>[1974817.835591]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:01 pm>[1974817.842186]  [<ffffffff81450996>] ? __alloc_skb+0x6f/0x168
<Jul/28 12:01 pm>[1974817.848138]  [<ffffffff8148f924>] ? tcp_current_mss+0x46/0x65
<Jul/28 12:01 pm>[1974817.854363]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:01 pm>[1974817.860969]  [<ffffffff8148677b>] ? tcp_sendmsg+0x1b2/0xa39
<Jul/28 12:01 pm>[1974817.867070]  [<ffffffff814471df>] ? sock_aio_write+0xdf/0xef
<Jul/28 12:01 pm>[1974817.873192]  [<ffffffff810ea282>] ? do_sync_write+0xc7/0x10d
<Jul/28 12:01 pm>[1974817.879338]  [<ffffffff8104a535>] ? sigprocmask+0x30/0xc8

<Jul/28 12:01 pm>[1974817.885207]  [<ffffffff812937db>] ? security_file_permissi<Jul/28 12:01 pm>
                 <Jul/28 12:01 pm>[1974817.892246]  [<ffffffff810eabb4>] ? vfs_write+0xc1/0x14a
<Jul/28 12:01 pm>[1974817.898043]  [<ffffffff810ead01>] ? sys_write+0x47/0x6e
<Jul/28 12:01 pm>[1974817.903730]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:01 pm>[1974827.037187] BUG: soft lockup - CPU#0 stuck for 61s! [kswapd0:608]
<Jul/28 12:01 pm>[1974827.043764] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:01 pm>[1974827.077972] CPU 0 
<Jul/28 12:01 pm>[1974827.080035] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:01 pm>[1974827.114575] 
<Jul/28 12:01 pm>[1974827.116438] Pid: 608, comm: kswapd0 Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:01 pm>[1974827.124565] RIP: 0010:[<ffffffff81028034>]  [<ffffffff81028034>] flush_tlb_others_ipi+0xb3/0xe6
<Jul/28 12:01 pm>[1974827.134005] RSP: 0018:ffff880623fe1890  EFLAGS: 00000206
<Jul/28 12:01 pm>[1974827.139781] RAX: 0000000000000018 RBX: ffff880623fe18d0 RCX: 0000000000000020
<Jul/28 12:01 pm>[1974827.147620] RDX: ffffffff81c16d40 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:01 pm>[1974827.155478] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81c16d40
<Jul/28 12:01 pm>[1974827.163318] R10: ffffffff81027fd3 R11: ffffffff812bba1d R12: ffff880623fe1860
<Jul/28 12:01 pm>[1974827.171168] R13: 0000000000000000 R14: 0000000000000246 R15: 0000000000000001
<Jul/28 12:01 pm>[1974827.178997] FS:  0000000000000000(0000) GS:ffff880002600000(0000) knlGS:0000000000000000
<Jul/28 12:01 pm>[1974827.187889] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:01 pm>[1974827.194105] CR2: 0000000000c7f108 CR3: 0000000001a8a000 CR4: 00000000000006f0
<Jul/28 12:01 pm>[1974827.201907] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:01 pm>[1974827.209773] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

<Jul/28 12:01 pm>[1974827.217597] Process kswapd0 (pid: 608, threadinfo ffff880623fe0000, task ffff880623f5b090)
<Jul/28 12:01 pm>[1974827.226581] Stack:
                 <Jul/28 12:01 pm>[1974827.228998]  ffff880500000000 ffff880c2363dc68 ffffffff81c16d10 ffff880c2363d940
<Jul/28 12:01 pm>[1974827.236845] <0> ffff880c2363dc68 00007f02a2136000 ffff880662be36b0 ffff880623fe1af0
<Jul/28 12:01 pm>[1974827.245299] <0> ffff880623fe1900 ffffffff810281b2 ffffffff810cc352 00000000ffffffff
<Jul/28 12:01 pm>[1974827.254130] Call Trace:
                 <Jul/28 12:01 pm>[1974827.257011]  [<ffffffff810281b2>] ? flush_tlb_page+0x6e/0x76
<Jul/28 12:01 pm>[1974827.263168]  [<ffffffff810cc352>] ? page_check_address+0xb0/0x106
<Jul/28 12:01 pm>[1974827.269789]  [<ffffffff81027599>] ? ptep_clear_flush_young+0x20/0x26
<Jul/28 12:01 pm>[1974827.276636]  [<ffffffff810cc62f>] ? page_referenced_one+0x65/0xe5
<Jul/28 12:01 pm>[1974827.283240]  [<ffffffff812bba1d>] ? prio_tree_next+0x208/0x22b
<Jul/28 12:01 pm>[1974827.289583]  [<ffffffff810cd6f1>] ? page_referenced+0x2e0/0x335
<Jul/28 12:01 pm>[1974827.295981]  [<ffffffff810e5fb2>] ? mem_cgroup_del_lru_list+0x7b/0x8e
<Jul/28 12:01 pm>[1974827.302942]  [<ffffffff810b6e86>] ? isolate_pages_global+0x1c6/0x23f
<Jul/28 12:01 pm>[1974827.309770]  [<ffffffff810b7ab2>] ? shrink_active_list+0x1d3/0x2a0
<Jul/28 12:01 pm>[1974827.316434]  [<ffffffff810b8003>] ? shrink_zone+0x484/0xb86
<Jul/28 12:01 pm>[1974827.322495]  [<ffffffff812734b7>] ? xfs_perag_get+0x29/0x99
<Jul/28 12:01 pm>[1974827.328588]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:01 pm>[1974827.334140]  [<ffffffff810b931a>] ? kswapd+0x61a/0x8bb
<Jul/28 12:01 pm>[1974827.339744]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:01 pm>[1974827.346449]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Jul/28 12:01 pm>[1974827.353390]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb

<Jul/28 12:01 pm>[1974827.358831]  [<ffffffff810546e2>] ? kthread+0x7d/0x85
<Jul/28 12:01 pm>[1974827.364346]  [<ffffffff81003794>] ? kernel_thread_helper+0x4/0x10
<Jul/28 12:01 pm>[1974827.370972]  [<ffffffff81054665>] ? kthread+0x0/0x85
                 <Jul/28 12:01 pm>[1974827.376394]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Jul/28 12:01 pm>[1974827.383004] Code: d0 48 23 02 48 85 c0 49 89 87 40 6d c1 81 74 20 48 8b 05 60 b8 af 00 41 8d b6 f0 00 00 00 48 89 df ff 90 e0 00 00 00 eb 02 f3 90 <83> 3b 00 75 f9 48 8b 7d d0 49 c7 87 00 6d c1 81 00 00 00 00 49 
<Jul/28 12:01 pm>[1974827.404369] Call Trace:
<Jul/28 12:01 pm>[1974827.407248]  [<ffffffff810281b2>] ? flush_tlb_page+0x6e/0x76
<Jul/28 12:01 pm>[1974827.413406]  [<ffffffff810cc352>] ? page_check_address+0xb0/0x106
<Jul/28 12:01 pm>[1974827.420029]  [<ffffffff81027599>] ? ptep_clear_flush_young+0x20/0x26
<Jul/28 12:01 pm>[1974827.426876]  [<ffffffff810cc62f>] ? page_referenced_one+0x65/0xe5
<Jul/28 12:01 pm>[1974827.433470]  [<ffffffff812bba1d>] ? prio_tree_next+0x208/0x22b
<Jul/28 12:01 pm>[1974827.444311]  [<ffffffff810cd6f1>] ? page_referenced+0x2e0/0x335
<Jul/28 12:01 pm>[1974827.450741]  [<ffffffff810e5fb2>] ? mem_cgroup_del_lru_list+0x7b/0x8e
<Jul/28 12:01 pm>[1974827.457685]  [<ffffffff810b6e86>] ? isolate_pages_global+0x1c6/0x23f
<Jul/28 12:01 pm>[1974827.464538]  [<ffffffff810b7ab2>] ? shrink_active_list+0x1d3/0x2a0
<Jul/28 12:01 pm>[1974827.471187]  [<ffffffff810b8003>] ? shrink_zone+0x484/0xb86
<Jul/28 12:01 pm>[1974827.477262]  [<ffffffff812734b7>] ? xfs_perag_get+0x29/0x99
<Jul/28 12:01 pm>[1974827.483312]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:01 pm>[1974827.488852]  [<ffffffff810b931a>] ? kswapd+0x61a/0x8bb
<Jul/28 12:01 pm>[1974827.494461]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:01 pm>[1974827.501165]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Jul/28 12:01 pm>[1974827.508090]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb

<Jul/28 12:01 pm>[1974827.513530]  [<ffffffff810546e2>] ? kthread+0x7d/0x85
<Jul/28 12:01 pm>[1974827.519081]  [<ffffffff81003794>] ? kernel_thread_helper+0x4/0x10
<Jul/28 12:01 pm>[1974827.525620]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Jul/28 12:01 pm>[1974827.531079]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Jul/28 12:02 pm>[1974868.914516] BUG: soft lockup - CPU#1 stuck for 61s! [sshd:9057]
<Jul/28 12:02 pm>[1974868.920929] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:02 pm>[1974868.955124] CPU 1 
<Jul/28 12:02 pm>[1974868.957202] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:02 pm>[1974868.991698] 
<Jul/28 12:02 pm>[1974868.993571] Pid: 9057, comm: sshd Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:02 pm>[1974869.001517] RIP: 0010:[<ffffffff8106a4e2>]  [<ffffffff8106a4e2>] smp_call_function_many+0x18e/0x1a5
<Jul/28 12:02 pm>[1974869.011328] RSP: 0018:ffff880c18747b20  EFLAGS: 00000202
<Jul/28 12:02 pm>[1974869.017104] RAX: 0000000000000020 RBX: ffff880c18747b60 RCX: 0000000000000020
<Jul/28 12:02 pm>[1974869.024952] RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:02 pm>[1974869.032783] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81b24210
<Jul/28 12:02 pm>[1974869.040598] R10: ffffffff812734b7 R11: 0000000000000286 R12: ffff880c18747ae0
<Jul/28 12:02 pm>[1974869.048473] R13: 0000000000000046 R14: ffff880c18747ae0 R15: 0000000000000000
<Jul/28 12:02 pm>[1974869.056279] FS:  00007f23edbf76f0(0000) GS:ffff880002620000(0000) knlGS:0000000000000000
<Jul/28 12:02 pm>[1974869.065085] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:02 pm>[1974869.071345] CR2: 00007fffec51b310 CR3: 0000000c239bf000 CR4: 00000000000006e0
<Jul/28 12:02 pm>[1974869.079195] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:02 pm>[1974869.087026] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

<Jul/28 12:02 pm>[1974869.094852] Process sshd (pid: 9057, threadinfo ffff880c18746000, task ffff880c187a0000)
<Jul/28 12:02 pm>[1974869.103704] Stack:
<Jul/28 12:02 pm>[1974869.106114]  01ff880c18747ba0 0000000000000000 ffffffff810b086c 0000000000000040
<Jul/28 12:02 pm>[1974869.113986] <0> 0000000000000000 ffffffff810b086c 0000000000000003 0000000000000002
<Jul/28 12:02 pm>[1974869.122422] <0> ffff880c18747b70 ffffffff8106a519 ffff880c18747ba0 ffffffff81040d28
<Jul/28 12:02 pm>[1974869.131261] Call Trace:
<Jul/28 12:02 pm>[1974869.134127]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974869.140456]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974869.146776]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:02 pm>[1974869.153242]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:02 pm>[1974869.159124]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:02 pm>[1974869.165803]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:02 pm>[1974869.172832]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:02 pm>[1974869.178360]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:02 pm>[1974869.184966]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:02 pm>[1974869.190667]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:02 pm>[1974869.196826]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974869.202886]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974869.208947]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:02 pm>[1974869.215300]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974869.221388]  [<ffffffff81063516>] ? lockdep_init_map+0x9f/0x52e
<Jul/28 12:02 pm>[1974869.227835]  [<ffffffff812c4f51>] ? __raw_spin_lock_init+0x31/0x54

<Jul/28 12:02 pm>[1974869.234576]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:02 pm>[1974869.240290]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:02 pm>[1974869.246617]  [<ffffffff810e8850>] ? fd_install+0x30/0x5f
<Jul/28 12:02 pm>[1974869.252440]  [<ffffffff810e8876>] ? fd_install+0x56/0x5f
<Jul/28 12:02 pm>[1974869.258269]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:02 pm>[1974869.263973]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:02 pm>[1974869.269781]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:02 pm>[1974869.276482] Code: c7 d0 21 a0 81 4c 89 60 08 4c 89 25 f9 7c 99 00 e8 59 51 4b 00 0f ae f0 48 89 df ff 15 f0 d1 a2 00 80 7d c7 00 75 04 eb 0a f3 90 <41> f6 44 24 20 01 75 f6 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f 
<Jul/28 12:02 pm>[1974869.297728] Call Trace:
<Jul/28 12:02 pm>[1974869.300581]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974869.306928]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974869.313275]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:02 pm>[1974869.319707]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:02 pm>[1974869.325625]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:02 pm>[1974869.332336]  [<ffffffff810b306c>] ? __alloc_pages_nodemask<Jul/28 12:02 pm>
                 <Jul/28 12:02 pm>[1974869.339388]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:02 pm>[1974869.344912]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:02 pm>[1974869.351500]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:02 pm>[1974869.357211]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:02 pm>[1974869.363377]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974869.369419]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974869.375487]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2

<Jul/28 12:02 pm>[1974869.381841]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974869.387917]  [<ffffffff81063516>] ? lockdep_init_map+0x9f/0x52e
<Jul/28 12:02 pm>[1974869.394350]  [<ffffffff812c4f51>] ? __raw_spin_lock_init+0x31/0x54
<Jul/28 12:02 pm>[1974869.401024]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:02 pm>[1974869.406754]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:02 pm>[1974869.413102]  [<ffffffff810e8850>] ? fd_install+0x30/0x5f
<Jul/28 12:02 pm>[1974869.418879]  [<ffffffff810e8876>] ? fd_install+0x56/0x5f
<Jul/28 12:02 pm>[1974869.424666]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:02 pm>[1974869.430404]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:02 pm>[1974869.436203]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:02 pm>[1974881.798686] BUG: soft lockup - CPU#3 stuck for 61s! [tcsh:3708]
<Jul/28 12:02 pm>[1974881.805085] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:02 pm>[1974881.839350] CPU 3 
<Jul/28 12:02 pm>[1974881.841420] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:02 pm>[1974881.876018] 
<Jul/28 12:02 pm>[1974881.877892] Pid: 3708, comm: tcsh Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:02 pm>[1974881.885837] RIP: 0010:[<ffffffff8106a4e2>]  [<ffffffff8106a4e2>] smp_call_function_many+0x18e/0x1a5
<Jul/28 12:02 pm>[1974881.895675] RSP: 0018:ffff880347d45b20  EFLAGS: 00000202
<Jul/28 12:02 pm>[1974881.901477] RAX: 0000000000000020 RBX: ffff880347d45b60 RCX: 0000000000000020
<Jul/28 12:02 pm>[1974881.909350] RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:02 pm>[1974881.917208] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81b24210
<Jul/28 12:02 pm>[1974881.925092] R10: ffffffff812734b7 R11: 0000000000000286 R12: ffff880347d45ae0

<Jul/28 12:02 pm>[1974881.932957] R13: 0000000000000046 R14: ffff880347d45ae0 R15: 0000000000000000
<Jul/28 12:02 pm>[1974881.940815] FS:  00007ff01d3fc6f0(0000) GS:ffff880002660000(0000) knlGS:0000000000000000
<Jul/28 12:02 pm>[1974881.949636] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:02 pm>[1974881.955832] CR2: 000000000065bbf8 CR3: 00000005ea477000 CR4: 00000000000006e0
<Jul/28 12:02 pm>[1974881.963708] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:02 pm>[1974881.971512] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:02 pm>[1974881.979360] Process tcsh (pid: 3708, threadinfo ffff880347d44000, task ffff8800b3493090)
<Jul/28 12:02 pm>[1974881.988173] Stack:
<Jul/28 12:02 pm>[1974881.990607]  01ff880347d45ba0 0000000000000000 ffffffff810b086c 0000000000000040
<Jul/28 12:02 pm>[1974881.998462] <0> 0000000000000000 ffffffff810b086c 0000000000000003 0000000000000002
<Jul/28 12:02 pm>[1974882.006959] <0> ffff880347d45b70 ffffffff8106a519 ffff880347d45ba0 ffffffff81040d28
<Jul/28 12:02 pm>[1974882.015789] Call Trace:
<Jul/28 12:02 pm>[1974882.018655]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974882.024991]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974882.031321]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:02 pm>[1974882.037701]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:02 pm>[1974882.043583]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:02 pm>[1974882.050303]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:02 pm>[1974882.057306]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:02 pm>[1974882.063919]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:02 pm>[1974882.069611]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:02 pm>[1974882.075780]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b

<Jul/28 12:02 pm>[1974882.082150]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974882.088191]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974882.094239]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:02 pm>[1974882.100604]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974882.106656]  [<ffffffff810c5179>] ? handle_mm_fault+0x6e3/0x73e
<Jul/28 12:02 pm>[1974882.113061]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:02 pm>[1974882.118564]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:02 pm>[1974882.124296]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:02 pm>[1974882.130637]  [<ffffffff810f6798>] ? set_close_on_exec+0x30/0x85
<Jul/28 12:02 pm>[1974882.137008]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Jul/28 12:02 pm>[1974882.143842]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:02 pm>[1974882.154095]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:02 pm>[1974882.159893]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:02 pm>[1974882.166593] Code: c7 d0 21 a0 81 4c 89 60 08 4c 89 25 f9 7c 99 00 e8 59 51 4b 00 0f ae f0 48 89 df ff 15 f0 d1 a2 00 80 7d c7 00 75 04 eb 0a f3 90 <41> f6 44 24 20 01 75 f6 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f 
<Jul/28 12:02 pm>[1974882.187916] Call Trace:
<Jul/28 12:02 pm>[1974882.190806]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974882.197154]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974882.203464]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:02 pm>[1974882.209913]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:02 pm>[1974882.215772]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:02 pm>[1974882.222456]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:02 pm>[1974882.229521]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f

<Jul/28 12:02 pm>[1974882.236098]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:02 pm>[1974882.241790]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:02 pm>[1974882.247966]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:02 pm>[1974882.254302]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974882.260359]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974882.266409]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:02 pm>[1974882.272776]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:02 pm>[1974882.278845]  [<ffffffff810c5179>] ? handle_mm_fault+0x6e3/0x73e
<Jul/28 12:02 pm>[1974882.285258]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:02 pm>[1974882.290814]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:02 pm>[1974882.296486]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:02 pm>[1974882.302832]  [<ffffffff810f6798>] ? set_close_on_exec+0x30/0x85
<Jul/28 12:02 pm>[1974882.309280]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Jul/28 12:02 pm>[1974882.316154]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:02 pm>[1974882.321873]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:02 pm>[1974882.327671]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:02 pm>[1974882.872772] BUG: soft lockup - CPU#5 stuck for 61s! [sshd:25093]
<Jul/28 12:02 pm>[1974882.879273] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:02 pm>[1974882.913336] CPU 5 
<Jul/28 12:02 pm>[1974882.915401] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:02 pm>[1974882.949869] 
<Jul/28 12:02 pm>[1974882.951743] Pid: 25093, comm: sshd Tainted: G      D W  2.6.34.4 #3 S7010/empty

<Jul/28 12:02 pm>[1974882.959745] RIP: 0010:[<ffffffff8106a4e8>]  [<ffffffff8106a4e8>] smp_call_function_many+0x194/0x1a5
<Jul/28 12:02 pm>[1974882.969518] RSP: 0018:ffff88035da9d978  EFLAGS: 00000202
<Jul/28 12:02 pm>[1974882.975293] RAX: 0000000000000020 RBX: ffff88035da9d9b8 RCX: 0000000000000020
<Jul/28 12:02 pm>[1974882.983087] RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:02 pm>[1974882.990920] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81b24210
<Jul/28 12:02 pm>[1974882.998743] R10: ffffffff812734b7 R11: 0000000000000286 R12: 0000000000000046
<Jul/28 12:02 pm>[1974883.006548] R13: ffff88035da9d938 R14: 0000000000000000 R15: 0000000000000046
<Jul/28 12:02 pm>[1974883.014355] FS:  00007f71cc38c6f0(0000) GS:ffff8800026a0000(0000) knlGS:0000000000000000
<Jul/28 12:02 pm>[1974883.023142] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:02 pm>[1974883.029373] CR2: 0000000000cc8058 CR3: 00000002d5944000 CR4: 00000000000006e0
<Jul/28 12:02 pm>[1974883.037201] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:02 pm>[1974883.045017] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:02 pm>[1974883.052823] Process sshd (pid: 25093, threadinfo ffff88035da9c000, task ffff880335739030)
<Jul/28 12:02 pm>[1974883.061739] Stack:
<Jul/28 12:02 pm>[1974883.064136]  01ff88035da9d9f8 0000000000000000 ffffffff810b086c 0000000000000040
<Jul/28 12:02 pm>[1974883.071935] <0> 0000000000000000 ffffffff810b086c 00000000<Jul/28 12:02 pm>00000002
                 <Jul/28 12:02 pm>[1974883.080370] <0> ffff88035da9d9c8 ffffffff8106a519 ffff88035da9d9f8 ffffffff81040d28
<Jul/28 12:02 pm>[1974883.089261] Call Trace:
<Jul/28 12:02 pm>[1974883.092116]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974883.098385]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974883.104705]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:02 pm>[1974883.111111]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c

<Jul/28 12:02 pm>[1974883.116986]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:02 pm>[1974883.123680]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:02 pm>[1974883.130716]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:02 pm>[1974883.137320]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:02 pm>[1974883.143020]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:02 pm>[1974883.149178]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:02 pm>[1974883.155784]  [<ffffffff810e06b3>] ? __kmalloc_node_track_caller+0x9f/0x114
<Jul/28 12:02 pm>[1974883.163170]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:02 pm>[1974883.169771]  [<ffffffff81450996>] ? __alloc_skb+0x6f/0x168
<Jul/28 12:02 pm>[1974883.175750]  [<ffffffff8148f924>] ? tcp_current_mss+0x46/0x65
<Jul/28 12:02 pm>[1974883.182010]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:02 pm>[1974883.188598]  [<ffffffff8148677b>] ? tcp_sendmsg+0x1b2/0xa39
<Jul/28 12:02 pm>[1974883.194631]  [<ffffffff814471df>] ? sock_aio_write+0xdf/0xef
<Jul/28 12:02 pm>[1974883.200768]  [<ffffffff810ea282>] ? do_sync_write+0xc7/0x10d
<Jul/28 12:02 pm>[1974883.206916]  [<ffffffff8104a535>] ? sigprocmask+0x30/0xc8
<Jul/28 12:02 pm>[1974883.212801]  [<ffffffff812937db>] ? security_file_permission+0x11/0x13
<Jul/28 12:02 pm>[1974883.219842]  [<ffffffff810eabb4>] ? vfs_write+0xc1/0x14a
<Jul/28 12:02 pm>[1974883.225611]  [<ffffffff810ead01>] ? sys_write+0x47/0x6e
<Jul/28 12:02 pm>[1974883.231292]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:02 pm>[1974883.237975] Code: 89 60 08 4c 89 25 f9 7c 99 00 e8 59 51 4b 00 0f ae f0 48 89 df ff 15 f0 d1 a2 00 80 7d c7 00 75 04 eb 0a f3 90 41 f6 44 24 20 01 <75> f6 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f c9 c3 48 89 f8 55 
<Jul/28 12:02 pm>[1974883.259253] Call Trace:
<Jul/28 12:02 pm>[1974883.262106]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13

<Jul/28 12:02 pm>[1974883.268427]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:02 pm>[1974883.274760]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:02 pm>[1974883.281171]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:02 pm>[1974883.287021]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:02 pm>[1974883.293671]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:02 pm>[1974883.300709]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:02 pm>[1974883.307295]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:02 pm>[1974883.312971]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:02 pm>[1974883.319101]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:02 pm>[1974883.325690]  [<ffffffff810e06b3>] ? __kmalloc_node_track_caller+0x9f/0x114
<Jul/28 12:02 pm>[1974883.333083]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:02 pm>[1974883.339642]  [<ffffffff81450996>] ? __alloc_skb+0x6f/0x168
<Jul/28 12:02 pm>[1974883.345612]  [<ffffffff8148f924>] ? tcp_current_mss+0x46/0x65
<Jul/28 12:02 pm>[1974883.351820]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:02 pm>[1974883.358402]  [<ffffffff8148677b>] ? tcp_sendmsg+0x1b2/0xa39
<Jul/28 12:02 pm>[1974883.364440]  [<ffffffff814471df>] ? sock_aio_write+0xdf/0xef
<Jul/28 12:02 pm>[1974883.370572]  [<ffffffff810ea282>] ? do_sync_write+0xc7/0x1<Jul/28 12:02 pm>
                 <Jul/28 12:02 pm>[1974883.376703]  [<ffffffff8104a535>] ? sigprocmask+0x30/0xc8
<Jul/28 12:02 pm>[1974883.382598]  [<ffffffff812937db>] ? security_file_permission+0x11/0x13
<Jul/28 12:02 pm>[1974883.389627]  [<ffffffff810eabb4>] ? vfs_write+0xc1/0x14a
<Jul/28 12:02 pm>[1974883.395387]  [<ffffffff810ead01>] ? sys_write+0x47/0x6e
<Jul/28 12:02 pm>[1974883.401067]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b

<Jul/28 12:02 pm>[1974892.537943] BUG: soft lockup - CPU#0 stuck for 61s! [kswapd0:608]
<Jul/28 12:02 pm>[1974892.544481] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:02 pm>[1974892.578638] CPU 0 
<Jul/28 12:02 pm>[1974892.580682] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:02 pm>[1974892.615187] 
<Jul/28 12:02 pm>[1974892.617069] Pid: 608, comm: kswapd0 Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:02 pm>[1974892.625188] RIP: 0010:[<ffffffff81028034>]  [<ffffffff81028034>] flush_tlb_others_ipi+0xb3/0xe6
<Jul/28 12:02 pm>[1974892.634661] RSP: 0018:ffff880623fe1890  EFLAGS: 00000206
<Jul/28 12:02 pm>[1974892.640447] RAX: 0000000000000018 RBX: ffff880623fe18d0 RCX: 0000000000000020
<Jul/28 12:02 pm>[1974892.648319] RDX: ffffffff81c16d40 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:02 pm>[1974892.656134] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81c16d40
<Jul/28 12:02 pm>[1974892.663991] R10: ffffffff81027fd3 R11: ffffffff812bba1d R12: ffff880623fe1860
<Jul/28 12:02 pm>[1974892.671813] R13: 0000000000000000 R14: 0000000000000246 R15: 0000000000000001
<Jul/28 12:02 pm>[1974892.679681] FS:  0000000000000000(0000) GS:ffff880002600000(0000) knlGS:0000000000000000
<Jul/28 12:02 pm>[1974892.688483] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:02 pm>[1974892.694725] CR2: 0000000000c7f108 CR3: 0000000001a8a000 CR4: 00000000000006f0
<Jul/28 12:02 pm>[1974892.702586] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:02 pm>[1974892.710462] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:02 pm>[1974892.718277] Process kswapd0 (pid: 608, threadinfo ffff880623fe0000, task ffff880623f5b090)
<Jul/28 12:02 pm>[1974892.727263] Stack:
<Jul/28 12:02 pm>[1974892.729697]  ffff880500000000 ffff880c2363dc68 ffffffff81c16d10 ffff880c2363d940
<Jul/28 12:02 pm>[1974892.737501] <0> ffff880c2363dc68 00007f02a2136000 ffff880662be36b0 ffff880623fe1af0

<Jul/28 12:02 pm>[1974892.746007] <0> ffff880623fe1900 ffffffff810281b2 ffffffff810cc352 00000000ffffffff
<Jul/28 12:02 pm>[1974892.754828] Call Trace:
<Jul/28 12:02 pm>[1974892.757709]  [<ffffffff810281b2>] ? flush_tlb_page+0x6e/0x76
<Jul/28 12:02 pm>[1974892.763874]  [<ffffffff810cc352>] ? page_check_address+0xb0/0x106
<Jul/28 12:02 pm>[1974892.770452]  [<ffffffff81027599>] ? ptep_clear_flush_young+0x20/0x26
<Jul/28 12:02 pm>[1974892.777327]  [<ffffffff810cc62f>] ? page_referenced_one+0x65/0xe5
<Jul/28 12:02 pm>[1974892.783912]  [<ffffffff812bba1d>] ? prio_tree_next+0x208/0x22b
<Jul/28 12:02 pm>[1974892.790212]  [<ffffffff810cd6f1>] ? page_referenced+0x2e0/0x335
<Jul/28 12:02 pm>[1974892.796636]  [<ffffffff810e5fb2>] ? mem_cgroup_del_lru_list+0x7b/0x8e
<Jul/28 12:02 pm>[1974892.808101]  [<ffffffff810b6e86>] ? isolate_pages_global+0x1c6/0x23f
<Jul/28 12:02 pm>[1974892.814930]  [<ffffffff810b7ab2>] ? shrink_active_list+0x1d3/0x2a0
<Jul/28 12:02 pm>[1974892.821602]  [<ffffffff810b8003>] ? shrink_zone+0x484/0xb86
<Jul/28 12:02 pm>[1974892.827688]  [<ffffffff812734b7>] ? xfs_perag_get+0x29/0x99
<Jul/28 12:02 pm>[1974892.833774]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:02 pm>[1974892.839311]  [<ffffffff810b931a>] ? kswapd+0x61a/0x8bb
<Jul/28 12:02 pm>[1974892.844939]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:02 pm>[1974892.851634]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Jul/28 12:02 pm>[1974892.858608]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb
<Jul/28 12:02 pm>[1974892.864043]  [<ffffffff810546e2>] ? kthread+0x7d/0x85
<Jul/28 12:02 pm>[1974892.869627]  [<ffffffff81003794>] ? kernel_thread_helper+0x4/0x10
<Jul/28 12:02 pm>[1974892.876201]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Jul/28 12:02 pm>[1974892.881632]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10

<Jul/28 12:02 pm>[1974892.888222] Code: d0 48 23 02 48 85 c0 49 89 87 40 6d c1 81 74 20 48 8b 05 60 b8 af 00 41 8d b6 f0 00 00 00 48 89 df ff 90 e0 00 00 00 eb 02 f3 90 <83> 3b 00 75 f9 48 8b 7d d0 49 c7 87 00 6d c1 81 00 00 00 00 49 
<Jul/28 12:02 pm>[1974892.909580] Call Trace:
<Jul/28 12:02 pm>[1974892.912467]  [<ffffffff810281b2>] ? flush_tlb_page+0x6e/0x76
<Jul/28 12:02 pm>[1974892.918643]  [<ffffffff810cc352>] ? page_check_address+0xb0/0x106
<Jul/28 12:02 pm>[1974892.925275]  [<ffffffff81027599>] ? ptep_clear_flush_young+0x20/0x26
<Jul/28 12:02 pm>[1974892.932157]  [<ffffffff810cc62f>] ? page_referenced_one+0x65/0xe5
<Jul/28 12:02 pm>[1974892.938726]  [<ffffffff812bba1d>] ? prio_tree_next+0x208/0x22b
<Jul/28 12:02 pm>[1974892.945079]  [<ffffffff810cd6f1>] ? page_referenced+0x2e0/0x335
<Jul/28 12:02 pm>[1974892.951486]  [<ffffffff810e5fb2>] ? mem_cgroup_del_lru_list+0x7b/0x8e
<Jul/28 12:02 pm>[1974892.958479]  [<ffffffff810b6e86>] ? isolate_pages_global+0x1c6/0x23f
<Jul/28 12:02 pm>[1974892.965394]  [<ffffffff810b7ab2>] ? shrink_active_list+0x1d3/0x2a0
<Jul/28 12:02 pm>[1974892.972119]  [<ffffffff810b8003>] ? shrink_zone+0x484/0xb86
<Jul/28 12:02 pm>[1974892.978177]  [<ffffffff812734b7>] ? xfs_perag_get+0x29/0x99
<Jul/28 12:02 pm>[1974892.984253]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:02 pm>[1974892.989783]  [<ffffffff810b931a>] ? kswapd+0x61a/0x8bb
<Jul/28 12:02 pm>[1974892.995428]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:02 pm>[1974893.002158]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Jul/28 12:02 pm>[1974893.009091]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb
<Jul/28 12:02 pm>[1974893.014540]  [<ffffffff810546e2>] ? kthread+0x7d/0x85
<Jul/28 12:02 pm>[1974893.020056]  [<ffffffff81003794>] ? kernel_thread_helper+0x4/0x10
<Jul/28 12:02 pm>[1974893.026664]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Jul/28 12:02 pm>[1974893.032115]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10

<Jul/28 12:03 pm>[1974934.411948] BUG: soft lockup - CPU#1 stuck for 61s! [sshd:9057]
<Jul/28 12:03 pm>[1974934.418363] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:03 pm>[1974934.452495] CPU 1 
<Jul/28 12:03 pm>[1974934.454558] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:03 pm>[1974934.489025] 
<Jul/28 12:03 pm>[1974934.490918] Pid: 9057, comm: sshd Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:03 pm>[1974934.498853] RIP: 0010:[<ffffffff8106a4e2>]  [<ffffffff8106a4e2>] smp_call_function_many+0x18e/0x1a5
<Jul/28 12:03 pm>[1974934.508638] RSP: 0018:ffff880c18747b20  EFLAGS: 00000202
<Jul/28 12:03 pm>[1974934.514425] RAX: 0000000000000020 RBX: ffff880c18747b60 RCX: 0000000000000020
<Jul/28 12:03 pm>[1974934.522287] RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:03 pm>[1974934.530121] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81b24210
<Jul/28 12:03 pm>[1974934.537970] R10: ffffffff812734b7 R11: 0000000000000286 R12: ffff880c18747ae0
<Jul/28 12:03 pm>[1974934.545776] R13: 0000000000000046 R14: ffff880c18747ae0 R15: 0000000000000000
<Jul/28 12:03 pm>[1974934.553645] FS:  00007f23edbf76f0(0000) GS:ffff880002620000(0000) knlGS:0000000000000000
<Jul/28 12:03 pm>[1974934.562455] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:03 pm>[1974934.568670] CR2: 00007fffec51b310 CR3: 0000000c239bf000 CR4: 00000000000006e0
<Jul/28 12:03 pm>[1974934.576507] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:03 pm>[1974934.584373] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:03 pm>[1974934.592223] Process sshd (pid: 9057, threadinfo ffff880c18746000, task ffff880c187a0000)
<Jul/28 12:03 pm>[1974934.601068] Stack:
                 <Jul/28 12:03 pm>[1974934.603486]  01ff880c18747ba0 0000000000000000 ffffffff810b086c 0000000000000040
<Jul/28 12:03 pm>[1974934.611321] <0> 0000000000000000 ffffffff810b086c 0000000000000003 0000000000000002

<Jul/28 12:03 pm>[1974934.619802] <0> ffff880c18747b70 ffffffff8106a519 ffff880c18747ba0 ffffffff81040d28
<Jul/28 12:03 pm>[1974934.628624] Call Trace:
<Jul/28 12:03 pm>[1974934.631507]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974934.637887]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974934.644210]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:03 pm>[1974934.650658]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:03 pm>[1974934.656542]  [<ffffffff810b6cc0>] ? isolate_pages_global+0<Jul/28 12:03 pm>
                 <Jul/28 12:03 pm>[1974934.663260]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:03 pm>[1974934.670281]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:03 pm>[1974934.675833]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:03 pm>[1974934.682449]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:03 pm>[1974934.688124]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:03 pm>[1974934.694309]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:03 pm>[1974934.700384]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:03 pm>[1974934.706443]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:03 pm>[1974934.712781]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:03 pm>[1974934.718831]  [<ffffffff81063516>] ? lockdep_init_map+0x9f/0x52e
<Jul/28 12:03 pm>[1974934.725250]  [<ffffffff812c4f51>] ? __raw_spin_lock_init+0x31/0x54
<Jul/28 12:03 pm>[1974934.731939]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:03 pm>[1974934.737652]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:03 pm>[1974934.744006]  [<ffffffff810e8850>] ? fd_install+0x30/0x5f
<Jul/28 12:03 pm>[1974934.749846]  [<ffffffff810e8876>] ? fd_install+0x56/0x5f

<Jul/28 12:03 pm>[1974934.755616]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:03 pm>[1974934.761328]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:03 pm>[1974934.767139]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:03 pm>[1974934.773811] Code: c7 d0 21 a0 81 4c 89 60 08 4c 89 25 f9 7c 99 00 e8 59 51 4b 00 0f ae f0 48 89 df ff 15 f0 d1 a2 00 80 7d c7 00 75 04 eb 0a f3 90 <41> f6 44 24 20 01 75 f6 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f 
<Jul/28 12:03 pm>[1974934.795184] Call Trace:
<Jul/28 12:03 pm>[1974934.798051]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974934.804403]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974934.810716]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:03 pm>[1974934.817113]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:03 pm>[1974934.823005]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:03 pm>[1974934.829691]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:03 pm>[1974934.836737]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:03 pm>[1974934.842231]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:03 pm>[1974934.848863]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:03 pm>[1974934.854572]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:03 pm>[1974934.860714]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:03 pm>[1974934.866739]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:03 pm>[1974934.872831]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:03 pm>[1974934.879187]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:03 pm>[1974934.885280]  [<ffffffff81063516>] ? lockdep_init_map+0x9f/0x52e
<Jul/28 12:03 pm>[1974934.891680]  [<ffffffff812c4f51>] ? __raw_spin_lock_init+0x31/0x54
<Jul/28 12:03 pm>[1974934.898377]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336

<Jul/28 12:03 pm>[1974934.904051]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:03 pm>[1974934.910401]  [<ffffffff810e8850>] ? fd_install+0x30/0x5f
<Jul/28 12:03 pm>[1974934.916179]  [<ffffffff810e8876>] ? fd_install+0x56/0x5f
<Jul/28 12:03 pm>[1974934.922010]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:03 pm>[1974934.927724]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:03 pm>[1974934.933558]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:03 pm>[1974947.296119] BUG: soft lockup - CPU#3 stuck for 61s! [tcsh:3708]
<Jul/28 12:03 pm>[1974947.302544] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:03 pm>[1974947.336710] CPU 3 
<Jul/28 12:03 pm>[1974947.338756] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:03 pm>[1974947.373425] 
                 <Jul/28 12:03 pm>[1974947.375315] Pid: 3708, comm: tcsh Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:03 pm>[1974947.383192] RIP: 0010:[<ffffffff8106a4e8>]  [<ffffffff8106a4e8>] smp_call_function_many+0x194/0x1a5
<Jul/28 12:03 pm>[1974947.392994] RSP: 0018:ffff880347d45b20  EFLAGS: 00000202
<Jul/28 12:03 pm>[1974947.398815] RAX: 0000000000000020 RBX: ffff880347d45b60 RCX: 0000000000000020
<Jul/28 12:03 pm>[1974947.406711] RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:03 pm>[1974947.414527] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81b24210
<Jul/28 12:03 pm>[1974947.422412] R10: ffffffff812734b7 R11: 0000000000000286 R12: ffff880347d45ae0
<Jul/28 12:03 pm>[1974947.430268] R13: 0000000000000046 R14: ffff880347d45ae0 R15: 0000000000000000
<Jul/28 12:03 pm>[1974947.438127] FS:  00007ff01d3fc6f0(0000) GS:ffff880002660000(0000) knlGS:0000000000000000
<Jul/28 12:03 pm>[1974947.446938] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:03 pm>[1974947.453162] CR2: 000000000065bbf8 CR3: 00000005ea477000 CR4: 00000000000006e0

<Jul/28 12:03 pm>[1974947.461026] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:03 pm>[1974947.468874] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:03 pm>[1974947.476741] Process tcsh (pid: 3708, threadinfo ffff880347d44000, task ffff8800b3493090)
<Jul/28 12:03 pm>[1974947.485593] Stack:
<Jul/28 12:03 pm>[1974947.488013]  01ff880347d45ba0 0000000000000000 ffffffff810b086c 0000000000000040
<Jul/28 12:03 pm>[1974947.495858] <0> 0000000000000000 ffffffff810b086c 0000000000000003 0000000000000002
<Jul/28 12:03 pm>[1974947.504297] <0> ffff880347d45b70 ffffffff8106a519 ffff880347d45ba0 ffffffff81040d28
<Jul/28 12:03 pm>[1974947.517683] Call Trace:
<Jul/28 12:03 pm>[1974947.520555]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974947.526884]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974947.533230]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:03 pm>[1974947.539636]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:03 pm>[1974947.545541]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:03 pm>[1974947.552199]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:03 pm>[1974947.559210]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:03 pm>[1974947.565811]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:03 pm>[1974947.571511]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:03 pm>[1974947.577722]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:03 pm>[1974947.584083]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:03 pm>[1974947.590178]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:03 pm>[1974947.596218]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:03 pm>[1974947.602591]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1

<Jul/28 12:03 pm>[1974947.608659]  [<ffffffff810c5179>] ? handle_mm_fault+0x6e3/0x73e
<Jul/28 12:03 pm>[1974947.615075]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:03 pm>[1974947.620594]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:03 pm>[1974947.626275]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:03 pm>[1974947.632640]  [<ffffffff810f6798>] ? set_close_on_exec+0x30/0x85
<Jul/28 12:03 pm>[1974947.639095]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Jul/28 12:03 pm>[1974947.645954]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:03 pm>[1974947.651674]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:03 pm>[1974947.657455]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:03 pm>[1974947.664122] Code: 89 60 08 4c 89 25 f9 7c 99 00 e8 59 51 4b 00 0f ae f0 48 89 df ff 15 f0 d1 a2 00 80 7d c7 00 75 04 eb 0a f3 90 41 f6 44 24 20 01 <75> f6 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f c9 c3 48 89 f8 55 
<Jul/28 12:03 pm>[1974947.685564] Call Trace:
<Jul/28 12:03 pm>[1974947.688444]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974947.694826]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974947.701171]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:03 pm>[1974947.707585]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:03 pm>[1974947.713446]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:03 pm>[1974947.720173]  [<ffffffff810b306c>] ? __alloc_pages_nodemask<Jul/28 12:03 pm>
                 <Jul/28 12:03 pm>[1974947.727210]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:03 pm>[1974947.733796]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:03 pm>[1974947.739533]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:03 pm>[1974947.745733]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:03 pm>[1974947.752087]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1

<Jul/28 12:03 pm>[1974947.758130]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:03 pm>[1974947.764188]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:03 pm>[1974947.770509]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:03 pm>[1974947.776576]  [<ffffffff810c5179>] ? handle_mm_fault+0x6e3/0x73e
<Jul/28 12:03 pm>[1974947.783035]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:03 pm>[1974947.788599]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:03 pm>[1974947.794348]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:03 pm>[1974947.800679]  [<ffffffff810f6798>] ? set_close_on_exec+0x30/0x85
<Jul/28 12:03 pm>[1974947.807083]  [<ffffffff8151ef6f>] ? lockdep_sys_exit_thunk+0x35/0x67
<Jul/28 12:03 pm>[1974947.813959]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
<Jul/28 12:03 pm>[1974947.819676]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:03 pm>[1974947.825485]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:03 pm>[1974948.370203] BUG: soft lockup - CPU#5 stuck for 61s! [sshd:25093]
<Jul/28 12:03 pm>[1974948.376688] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:03 pm>[1974948.410735] CPU 5 
<Jul/28 12:03 pm>[1974948.412797] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:03 pm>[1974948.447310] 
<Jul/28 12:03 pm>[1974948.449183] Pid: 25093, comm: sshd Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:03 pm>[1974948.457180] RIP: 0010:[<ffffffff8106a4e8>]  [<ffffffff8106a4e8>] smp_call_function_many+0x194/0x1a5
<Jul/28 12:03 pm>[1974948.466931] RSP: 0018:ffff88035da9d978  EFLAGS: 00000202
<Jul/28 12:03 pm>[1974948.472709] RAX: 0000000000000020 RBX: ffff88035da9d9b8 RCX: 0000000000000020
<Jul/28 12:03 pm>[1974948.480556] RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000

<Jul/28 12:03 pm>[1974948.488376] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81b24210
<Jul/28 12:03 pm>[1974948.496193] R10: ffffffff812734b7 R11: 0000000000000286 R12: 0000000000000046
<Jul/28 12:03 pm>[1974948.504007] R13: ffff88035da9d938 R14: 0000000000000000 R15: 0000000000000046
<Jul/28 12:03 pm>[1974948.511812] FS:  00007f71cc38c6f0(0000) GS:ffff8800026a0000(0000) knlGS:0000000000000000
<Jul/28 12:03 pm>[1974948.520640] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:03 pm>[1974948.526873] CR2: 0000000000cc8058 CR3: 00000002d5944000 CR4: 00000000000006e0
<Jul/28 12:03 pm>[1974948.534736] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:03 pm>[1974948.542587] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:03 pm>[1974948.550471] Process sshd (pid: 25093, threadinfo ffff88035da9c000, task ffff880335739030)
<Jul/28 12:03 pm>[1974948.559361] Stack:
<Jul/28 12:03 pm>[1974948.561769]  01ff88035da9d9f8 0000000000000000 ffffffff810b086c 0000000000000040
<Jul/28 12:03 pm>[1974948.569571] <0> 0000000000000000 ffffffff810b086c 0000000000000003 0000000000000002
<Jul/28 12:03 pm>[1974948.577991] <0> ffff88035da9d9c8 ffffffff8106a519 ffff88035da9d9f8 ffffffff81040d28
<Jul/28 12:03 pm>[1974948.586813] Call Trace:
<Jul/28 12:03 pm>[1974948.589679]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974948.595987]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974948.602310]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:03 pm>[1974948.608698]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:03 pm>[1974948.614556]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:03 pm>[1974948.621230]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:03 pm>[1974948.628261]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:03 pm>[1974948.634844]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b

<Jul/28 12:03 pm>[1974948.640523]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:03 pm>[1974948.646662]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:03 pm>[1974948.653259]  [<ffffffff810e06b3>] ? __kmalloc_node_track_caller+0x9f/0x114
<Jul/28 12:03 pm>[1974948.660599]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:03 pm>[1974948.667169]  [<ffffffff81450996>] ? __alloc_skb+0x6f/0x168
<Jul/28 12:03 pm>[1974948.673195]  [<ffffffff8148f924>] ? tcp_current_mss+0x46/0x65
<Jul/28 12:03 pm>[1974948.679407]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:03 pm>[1974948.685977]  [<ffffffff8148677b>] ? tcp_sendmsg+0x1b2/0xa39
<Jul/28 12:03 pm>[1974948.691994]  [<ffffffff814471df>] ? sock_aio_write+0xdf/0xef
<Jul/28 12:03 pm>[1974948.698158]  [<ffffffff810ea282>] ? do_sync_write+0xc7/0x10d
<Jul/28 12:03 pm>[1974948.704268]  [<ffffffff8104a535>] ? sigprocmask+0x30/0xc8
<Jul/28 12:03 pm>[1974948.710130]  [<ffffffff812937db>] ? security_file_permission+0x11/0x13
<Jul/28 12:03 pm>[1974948.717134]  [<ffffffff810eabb4>] ? vfs_write+0xc1/0x14a
<Jul/28 12:03 pm>[1974948.722929]  [<ffffffff810ead01>] ? sys_write+0x47/0x6e
<Jul/28 12:03 pm>[1974948.728627]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:03 pm>[1974948.735302] Code: 89 60 08 4c 89 25 f9 7c 99 00 e8 59 51 4b 00 0f ae f0 48 89 df ff 15 f0 d1 a2 00 80 7d c7 00 75 04 eb 0a f3 90 41 f6 44 <Jul/28 12:03 pm>48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f c9 c3 48 89 f8 55 
                 <Jul/28 12:03 pm>[1974948.756649] Call Trace:
<Jul/28 12:03 pm>[1974948.759504]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:03 pm>[1974948.765808]  [<ffffffff810b086c>] ? drain_local_pages+0x0/<Jul/28 12:03 pm>
                 <Jul/28 12:03 pm>[1974948.772127]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:03 pm>[1974948.778527]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:03 pm>[1974948.784408]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f

<Jul/28 12:03 pm>[1974948.791068]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:03 pm>[1974948.798059]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:03 pm>[1974948.804665]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:03 pm>[1974948.810374]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x4<Jul/28 12:03 pm>
                 <Jul/28 12:03 pm>[1974948.816492]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:03 pm>[1974948.823060]  [<ffffffff810e06b3>] ? __kmalloc_node_track_caller+0x9f/0x114
<Jul/28 12:03 pm>[1974948.830446]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:03 pm>[1974948.837049]  [<ffffffff81450996>] ? __alloc_skb+0x6f/0x168
<Jul/28 12:03 pm>[1974948.843011]  [<ffffffff8148f924>] ? tcp_current_mss+0x46/0x65
<Jul/28 12:03 pm>[1974948.849260]  [<ffffffff81486514>] ? sk_stream_alloc_skb+0x38/0xed
<Jul/28 12:03 pm>[1974948.855815]  [<ffffffff8148677b>] ? tcp_sendmsg+0x1b2/0xa39
<Jul/28 12:03 pm>[1974948.861855]  [<ffffffff814471df>] ? sock_aio_write+0xdf/0xef
<Jul/28 12:03 pm>[1974948.868021]  [<ffffffff810ea282>] ? do_sync_write+0xc7/0x10d
<Jul/28 12:03 pm>[1974948.874174]  [<ffffffff8104a535>] ? sigprocmask+0x30/0xc8
<Jul/28 12:03 pm>[1974948.880079]  [<ffffffff812937db>] ? security_file_permission+0x11/0x13
<Jul/28 12:03 pm>[1974948.887118]  [<ffffffff810eabb4>] ? vfs_write+0xc1/0x14a
<Jul/28 12:03 pm>[1974948.892895]  [<ffffffff810ead01>] ? sys_write+0x47/0x6e
<Jul/28 12:03 pm>[1974948.898602]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:03 pm>[1974958.035375] BUG: soft lockup - CPU#0 stuck for 61s! [kswapd0:608]
<Jul/28 12:03 pm>[1974958.041964] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:03 pm>[1974958.076122] CPU 0 
<Jul/28 12:03 pm>[1974958.078176] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis

<Jul/28 12:03 pm>[1974958.112676] 
<Jul/28 12:03 pm>[1974958.114568] Pid: 608, comm: kswapd0 Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:03 pm>[1974958.122670] RIP: 0010:[<ffffffff81028032>]  [<ffffffff81028032>] flush_tlb_others_ipi+0xb1/0xe6
<Jul/28 12:03 pm>[1974958.132162] RSP: 0018:ffff880623fe1890  EFLAGS: 00000206
<Jul/28 12:03 pm>[1974958.137974] RAX: 0000000000000018 RBX: ffff880623fe18d0 RCX: 0000000000000020
<Jul/28 12:03 pm>[1974958.145820] RDX: ffffffff81c16d40 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:03 pm>[1974958.153650] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81c16d40
<Jul/28 12:03 pm>[1974958.161492] R10: ffffffff81027fd3 R11: ffffffff812bba1d R12: ffff880623fe1860
<Jul/28 12:03 pm>[1974958.169367] R13: 0000000000000000 R14: 0000000000000246 R15: 0000000000000001
<Jul/28 12:03 pm>[1974958.177190] FS:  0000000000000000(0000) GS:ffff880002600000(0000) knlGS:0000000000000000
<Jul/28 12:03 pm>[1974958.186013] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:03 pm>[1974958.192280] CR2: 0000000000c7f108 CR3: 0000000001a8a000 CR4: 00000000000006f0
<Jul/28 12:03 pm>[1974958.204701] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:03 pm>[1974958.212564] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:03 pm>[1974958.220403] Process kswapd0 (pid: 608, threadinfo ffff880623fe0000, task ffff880623f5b090)
<Jul/28 12:03 pm>[1974958.229405] Stack:
<Jul/28 12:03 pm>[1974958.231805]  ffff880500000000 ffff880c2363dc68 ffffffff81c16d10 ffff880c2363d940
<Jul/28 12:03 pm>[1974958.239600] <0> ffff880c2363dc68 00007f02a2136000 ffff880662be36b0 ffff880623fe1af0
<Jul/28 12:03 pm>[1974958.248088] <0> ffff880623fe1900 ffffffff810281b2 ffffffff810cc352 00000000ffffffff
<Jul/28 12:03 pm>[1974958.256919] Call Trace:
<Jul/28 12:03 pm>[1974958.259782]  [<ffffffff810281b2>] ? flush_tlb_page+0x6e/0x76
<Jul/28 12:03 pm>[1974958.265966]  [<ffffffff810cc352>] ? page_check_address+0xb0/0x106

<Jul/28 12:03 pm>[1974958.272552]  [<ffffffff81027599>] ? ptep_clear_flush_young+0x20/0x26
<Jul/28 12:03 pm>[1974958.279417]  [<ffffffff810cc62f>] ? page_referenced_one+0x65/0xe5
<Jul/28 12:03 pm>[1974958.286021]  [<ffffffff812bba1d>] ? prio_tree_next+0x208/0x22b
<Jul/28 12:03 pm>[1974958.292364]  [<ffffffff810cd6f1>] ? page_referenced+0x2e0/0x335
<Jul/28 12:03 pm>[1974958.298772]  [<ffffffff810e5fb2>] ? mem_cgroup_del_lru_list+0x7b/0x8e
<Jul/28 12:03 pm>[1974958.305730]  [<ffffffff810b6e86>] ? isolate_pages_global+0x1c6/0x23f
<Jul/28 12:03 pm>[1974958.312558]  [<ffffffff810b7ab2>] ? shrink_active_list+0x1d3/0x2a0
<Jul/28 12:03 pm>[1974958.319266]  [<ffffffff810b8003>] ? shrink_zone+0x484/0xb86
<Jul/28 12:03 pm>[1974958.325308]  [<ffffffff812734b7>] ? xfs_perag_get+0x29/0x99
<Jul/28 12:03 pm>[1974958.331352]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:03 pm>[1974958.336862]  [<ffffffff810b931a>] ? kswapd+0x61a/0x8bb
<Jul/28 12:03 pm>[1974958.342484]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:03 pm>[1974958.349221]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Jul/28 12:03 pm>[1974958.356162]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb
<Jul/28 12:03 pm>[1974958.361621]  [<ffffffff810546e2>] ? kthread+0x7d/0x85
<Jul/28 12:03 pm>[1974958.367162]  [<ffffffff81003794>] ? kernel_thread_helper+0x4/0x10
<Jul/28 12:03 pm>[1974958.373735]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Jul/28 12:03 pm>[1974958.379167]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Jul/28 12:03 pm>[1974958.385793] Code: 48 f7 d0 48 23 02 48 85 c0 49 89 87 40 6d c1 81 74 20 48 8b 05 60 b8 af 00 41 8d b6 f0 00 00 00 48 89 df ff 90 e0 00 00 00 eb 02 <f3> 90 83 3b 00 75 f9 48 8b 7d d0 49 c7 87 00 6d c1 81 00 00 00 
<Jul/28 12:03 pm>[1974958.407132] Call Trace:
<Jul/28 12:03 pm>[1974958.409977]  [<ffffffff810281b2>] ? flush_tlb_page+0x6e/0x76
<Jul/28 12:03 pm>[1974958.416117]  [<ffffffff810cc352>] ? page_check_address+0xb0/0x106

<Jul/28 12:03 pm>[1974958.422750]  [<ffffffff81027599>] ? ptep_clear_flush_young+0x20/0x26
<Jul/28 12:03 pm>[1974958.429632]  [<ffffffff810cc62f>] ? page_referenced_one+0x65/0xe5
<Jul/28 12:03 pm>[1974958.436198]  [<ffffffff812bba1d>] ? prio_tree_next+0x208/0x22b
<Jul/28 12:03 pm>[1974958.442517]  [<ffffffff810cd6f1>] ? page_referenced+0x2e0/0x335
<Jul/28 12:03 pm>[1974958.448915]  [<ffffffff810e5fb2>] ? mem_cgroup_del_lru_lis<Jul/28 12:03 pm>
                 <Jul/28 12:03 pm>[1974958.455884]  [<ffffffff810b6e86>] ? isolate_pages_global+0x1c6/0x23f
<Jul/28 12:03 pm>[1974958.462757]  [<ffffffff810b7ab2>] ? shrink_active_list+0x1d3/0x2a0
<Jul/28 12:03 pm>[1974958.469456]  [<ffffffff810b8003>] ? shrink_zone+0x484/0xb86
<Jul/28 12:03 pm>[1974958.475513]  [<ffffffff812734b7>] ? xfs_perag_get+0x29/0x99
<Jul/28 12:03 pm>[1974958.481573]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:03 pm>[1974958.487111]  [<ffffffff810b931a>] ? kswapd+0x61a/0x8bb
<Jul/28 12:03 pm>[1974958.492731]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:03 pm>[1974958.499455]  [<ffffffff81054b6c>] ? autoremove_wake_function+0x0/0x38
<Jul/28 12:03 pm>[1974958.506455]  [<ffffffff810b8d00>] ? kswapd+0x0/0x8bb
<Jul/28 12:03 pm>[1974958.511938]  [<ffffffff810546e2>] ? kthread+0x7d/0x85
<Jul/28 12:03 pm>[1974958.517471]  [<ffffffff81003794>] ? kernel_thread_helper+0x4/0x10
<Jul/28 12:03 pm>[1974958.524060]  [<ffffffff81054665>] ? kthread+0x0/0x85
<Jul/28 12:03 pm>[1974958.529528]  [<ffffffff81003790>] ? kernel_thread_helper+0x0/0x10
<Jul/28 12:04 pm>[1974999.909379] BUG: soft lockup - CPU#1 stuck for 61s! [sshd:9057]
<Jul/28 12:04 pm>[1974999.915785] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis
<Jul/28 12:04 pm>[1974999.950057] CPU 1 
<Jul/28 12:04 pm>[1974999.952128] Modules linked in: w83793 hwmon_vid autofs4 e1000e i2c_i801 i2c_core iTCO_wdt ioatdma dca rtc_cmos tg3 libphy e1000 fuse dm_snapshot dm_mirror dm_region_hash dm_log scsi_wait_scan qla2xxx megaraid_sas megaraid_mbox megaraid_mm aacraid 3w_9xxx 3w_xxxx arcmsr aic7xxx aic79xx sr_mod sg ata_piix ahci sata_sis sata_nv sata_via pata_sl82c105 pata_sis

<Jul/28 12:04 pm>[1974999.986683] 
<Jul/28 12:04 pm>[1974999.988600] Pid: 9057, comm: sshd Tainted: G      D W  2.6.34.4 #3 S7010/empty
<Jul/28 12:04 pm>[1974999.996511] RIP: 0010:[<ffffffff8106a4e2>]  [<ffffffff8106a4e2>] smp_call_function_many+0x18e/0x1a5
<Jul/28 12:04 pm>[1975000.006303] RSP: 0018:ffff880c18747b20  EFLAGS: 00000202
<Jul/28 12:04 pm>[1975000.012116] RAX: 0000000000000020 RBX: ffff880c18747b60 RCX: 0000000000000020
<Jul/28 12:04 pm>[1975000.019979] RDX: 0000000000000000 RSI: 0000000000000020 RDI: 0000000000000000
<Jul/28 12:04 pm>[1975000.027845] RBP: ffffffff8100334e R08: 0000000000000020 R09: ffffffff81b24210
<Jul/28 12:04 pm>[1975000.035668] R10: ffffffff812734b7 R11: 0000000000000286 R12: ffff880c18747ae0
<Jul/28 12:04 pm>[1975000.043535] R13: 0000000000000046 R14: ffff880c18747ae0 R15: 0000000000000000
<Jul/28 12:04 pm>[1975000.051352] FS:  00007f23edbf76f0(0000) GS:ffff880002620000(0000) knlGS:0000000000000000
<Jul/28 12:04 pm>[1975000.060178] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Jul/28 12:04 pm>[1975000.066447] CR2: 00007fffec51b310 CR3: 0000000c239bf000 CR4: 00000000000006e0
<Jul/28 12:04 pm>[1975000.074292] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Jul/28 12:04 pm>[1975000.082175] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Jul/28 12:04 pm>[1975000.089999] Process sshd (pid: 9057, threadinfo ffff880c18746000, task ffff880c187a0000)
<Jul/28 12:04 pm>[1975000.098768] Stack:
<Jul/28 12:04 pm>[1975000.101167]  01ff880c18747ba0 0000000000000000 ffffffff810b086c 0000000000000040
<Jul/28 12:04 pm>[1975000.108978] <0> 0000000000000000 ffffffff810b086c 0000000000000003 0000000000000002
<Jul/28 12:04 pm>[1975000.117494] <0> ffff880c18747b70 ffffffff8106a519 ffff880c18747ba0 ffffffff81040d28
<Jul/28 12:04 pm>[1975000.126367] Call Trace:
<Jul/28 12:04 pm>[1975000.129249]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:04 pm>[1975000.135604]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13

<Jul/28 12:04 pm>[1975000.141923]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:04 pm>[1975000.148382]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:04 pm>[1975000.154257]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:04 pm>[1975000.160949]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:04 pm>[1975000.168005]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:04 pm>[1975000.173525]  [<ffffffff810da7be>] ? alloc_pages_current+0x96/0x9f
<Jul/28 12:04 pm>[1975000.180114]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:04 pm>[1975000.185868]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:04 pm>[1975000.192052]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:04 pm>[1975000.198128]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:04 pm>[1975000.204243]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:04 pm>[1975000.210537]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:04 pm>[1975000.216620]  [<ffffffff81063516>] ? lockdep_init_map+0x9f/0x52e
<Jul/28 12:04 pm>[1975000.223052]  [<ffffffff812c4f51>] ? __raw_spin_lock_init+0x31/0x54
<Jul/28 12:04 pm>[1975000.229751]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
                 <Jul/28 12:04 pm>[1975000.235473]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:04 pm>[1975000.241834]  [<ffffffff810e8850>] ? fd_install+0x30/0x5f
<Jul/28 12:04 pm>[1975000.247651]  [<ffffffff810e8876>] ? fd_install+0x56/0x5f
<Jul/28 12:04 pm>[1975000.253426]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25
                 <Jul/28 12:04 pm>[1975000.259097]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:04 pm>[1975000.264913]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b
<Jul/28 12:04 pm>[1975000.271642] Code: c7 d0 21 a0 81 4c 89 60 08 4c 89 25 f9 7c 99 00 e8 59 51 4b 00 0f ae f0 48 89 df ff 15 f0 d1 a2 00 80 7d c7 00 75 04 eb 0a f3 90 <41> f6 44 24 20 01 75 f6 48 83 c4 18 5b 41 5c 41 5d 41 5e 41 5f 

<Jul/28 12:04 pm>[1975000.292989] Call Trace:
<Jul/28 12:04 pm>[1975000.295868]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:04 pm>[1975000.302180]  [<ffffffff810b086c>] ? drain_local_pages+0x0/0x13
<Jul/28 12:04 pm>[1975000.308543]  [<ffffffff8106a519>] ? smp_call_function+0x20/0x24
<Jul/28 12:04 pm>[1975000.314952]  [<ffffffff81040d28>] ? on_each_cpu+0x18/0x2c
<Jul/28 12:04 pm>[1975000.320844]  [<ffffffff810b6cc0>] ? isolate_pages_global+0x0/0x23f
<Jul/28 12:04 pm>[1975000.327528]  [<ffffffff810b306c>] ? __alloc_pages_nodemask+0x44d/0x671
<Jul/28 12:04 pm>[1975000.334567]  [<ffffffff81058774>] ? up_read+0x1e/0x36
<Jul/28 12:04 pm>[1975000.340133]  [<ffffffff810da7be>] ? alloc_pages_current+0x<Jul/28 12:04 pm>
                 <Jul/28 12:04 pm>[1975000.346765]  [<ffffffff810df0c8>] ? new_slab+0x57/0x26b
<Jul/28 12:04 pm>[1975000.352510]  [<ffffffff810df4f8>] ? __slab_alloc+0x21c/0x45f
<Jul/28 12:04 pm>[1975000.358674]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:04 pm>[1975000.364732]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:04 pm>[1975000.370826]  [<ffffffff810dfb2f>] ? kmem_cache_alloc+0x6a/0xd2
<Jul/28 12:04 pm>[1975000.377129]  [<ffffffff810397ea>] ? copy_process+0xce/0xfc1
<Jul/28 12:04 pm>[1975000.383214]  [<ffffffff81063516>] ? lockdep_init_map+0x9f/0x52e
<Jul/28 12:04 pm>[1975000.389620]  [<ffffffff812c4f51>] ? __raw_spin_lock_init+0x31/0x54
<Jul/28 12:04 pm>[1975000.396295]  [<ffffffff8103a83d>] ? do_fork+0x160/0x336
<Jul/28 12:04 pm>[1975000.402023]  [<ffffffff8151f67a>] ? _raw_spin_unlock+0x26/0x2b
<Jul/28 12:04 pm>[1975000.408353]  [<ffffffff810e8850>] ? fd_install+0x30/0x5f
<Jul/28 12:04 pm>[1975000.414142]  [<ffffffff810e8876>] ? fd_install+0x56/0x5f
<Jul/28 12:04 pm>[1975000.419943]  [<ffffffff810098ba>] ? sys_clone+0x23/0x25

<Jul/28 12:04 pm>[1975000.425665]  [<ffffffff81002cd3>] ? stub_clone+0x13/0x20
<Jul/28 12:04 pm>[1975000.431499]  [<ffffffff810029ab>] ? system_call_fastpath+0x16/0x1b