ipp018-crash-20121206 – Pan-STARRS IPP

wiki:ipp018-crash-20121206

Context Navigation

from console:

<Dec/06 07:56 pm>ipp018 login: [1742907.002018] general protection fault: 0000 [#1] SMP 
<Dec/06 07:56 pm>[1742907.002380] last sysfs file: /sys/class/i2c-adapter/i2c-0/name
<Dec/06 07:56 pm>[1742907.002380] CPU 0 
<Dec/06 07:56 pm>[1742907.002380] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:56 pm>[1742907.002380] Pid: 1524, comm: rpciod/0 Tainted: G        W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:56 pm>[1742907.002380] RIP: 0010:[<ffffffff802488f9>]  [<ffffffff802488f9>] run_workqueue+0x73/0x1ed
<Dec/06 07:56 pm>[1742907.002380] RSP: 0018:ffff88033ccdbe80  EFLAGS: 00010087
<Dec/06 07:56 pm>[1742907.002380] RAX: 66fbf15517d3a542 RBX: ffff880323904580 RCX: 0000000000000000
<Dec/06 07:56 pm>[1742907.002380] RDX: a919dae7eefa4125 RSI: ffff8803239045b0 RDI: ffff88033d201100
<Dec/06 07:56 pm>[1742907.002380] RBP: ffff88033d201100 R08: 0000000000000001 R09: 0000000000000001
<Dec/06 07:56 pm>[1742907.002380] R10: 0000000000000000 R11: 0000000000000000 R12: ffff880323904578
<Dec/06 07:56 pm>[1742907.002380] R13: d6f684b2ef9f442d R14: ffffffff808b85a0 R15: 0000000000000000
<Dec/06 07:56 pm>[1742907.002380] FS:  0000000000000000(0000) GS:ffffffff80872d80(0000) knlGS:0000000000000000
<Dec/06 07:56 pm>[1742907.002380] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
<Dec/06 07:56 pm>[1742907.002380] CR2: 00007f186495d000 CR3: 00000002994a0000 CR4: 00000000000006e0

<Dec/06 07:56 pm>[1742907.002380] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:56 pm>[1742907.002380] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 07:56 pm>[1742907.002380] Process rpciod/0 (pid: 1524, threadinfo ffff88033ccda000, task ffff88033cc61e60)
<Dec/06 07:56 pm>[1742907.002380] Stack:
<Dec/06 07:56 pm>[1742907.002380]  ffff88033d201140 0000000000000046 b0de62e37a334368 d3ead54ec2973d26
<Dec/06 07:56 pm>[1742907.002380]  a1768b44857f2f7f ffff88033d201100 ffff88063e43bbf0 ffff88033d201100
<Dec/06 07:56 pm>[1742907.002380]  ffff88033d201140 ffff88063e43bbf0 ffffffff808b5d60 ffffffff80249511
<Dec/06 07:56 pm>[1742907.002380] Call Trace:
<Dec/06 07:56 pm>[1742907.002380]  [<ffffffff80249511>] ? worker_thread+0xd8/0xe3
<Dec/06 07:56 pm>[1742907.002380]  [<ffffffff8024bfda>] ? autoremove_wake_function+0x0/0x2e
<Dec/06 07:56 pm>[1742907.002380]  [<ffffffff80249439>] ? worker_thread+0x0/0xe3
<Dec/06 07:56 pm>[1742907.002380]  [<ffffffff8024becc>] ? kthread+0x47/0x73
<Dec/06 07:56 pm>[1742907.002380]  [<ffffffff80234852>] ? schedule_tail+0x27/0x5f
<Dec/06 07:56 pm>[1742907.002380]  [<ffffffff8020cd19>] ? child_rip+0xa/0x11
<Dec/06 07:56 pm>[1742907.002380]  [<ffffffff804fa23c>] ? menu_reflect+0x0/0x6d
<Dec/06 07:56 pm>[1742907.002380]  [<ffffffff8024be85>] ? kthread+0x0/0x73
<Dec/06 07:56 pm>[1742907.002380]  [<ffffffff8020cd0f>] ? child_rip+0x0/0x11
<Dec/06 07:56 pm>[1742907.002380] Code: 4c 8d 63 f8 48 8d 7c 24 10 48 8d 73 18 fc 4d 8b 6c 24 18 b9 06 00 00 00 f3 a5 4c 89 a5 80 00 00 00 48 89 ef 48 8b 43 08 48 8b 13 <48> 89 42 08 48 89 10 48 89 5b 08 48 89 1b e8 d1 ed 36 00 48 8b 
<Dec/06 07:56 pm>[1742907.002380] RIP  [<ffffffff802488f9>] run_workqueue+0x73/0x1ed
<Dec/06 07:56 pm>[1742907.002380]  RSP <ffff88033ccdbe80>
<Dec/06 07:56 pm>[1742907.002380] ---[ end trace 6e4e47cca4dd6312 ]---
<Dec/06 07:57 pm>[1742912.482643] BUG: spinlock lockup on CPU#2, lockd/7748, ffff88033d201100

<Dec/06 07:57 pm>[1742912.482643] Pid: 7748, comm: lockd Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:57 pm>[1742912.482643] Call Trace:
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff803a103f>] _raw_spin_lock+0xfb/0x122
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8024912b>] __cancel_work_timer+0x120/0x1a3
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff802490a1>] __cancel_work_timer+0x96/0x1a3
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8058a0f0>] xs_destroy+0x31/0x79
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff80588458>] xprt_destroy+0x0/0x7c
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff803995bf>] kref_put+0x41/0x4c
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff80587073>] rpc_free_client+0x0/0xd1
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff80587136>] rpc_free_client+0xc3/0xd1
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff803995bf>] kref_put+0x41/0x4c
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff80587144>] rpc_free_auth+0x0/0x40
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff803995bf>] kref_put+0x41/0x4c
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8058724e>] rpc_shutdown_client+0xca/0xd5
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff80586fc6>] rpc_call_sync+0x53/0x5b
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8037054a>] nsm_mon_unmon+0x158/0x16c
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff803995bf>] kref_put+0x41/0x4c
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff803705c4>] nsm_unmonitor+0x66/0xa2
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8036c8fa>] nlm_gc_hosts+0x126/0x18d
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8036d172>] nlm_lookup_host+0x32/0x4f9
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8036d746>] nlmsvc_lookup_host+0x10d/0x119
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff80372235>] nlm4svc_retrieve_args+0x49/0xe3

<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff803a0f26>] __spin_lock_init+0x29/0x47
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8037272e>] nlm4svc_proc_lock+0x4b/0xda
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff80371d04>] nlm4svc_decode_lockargs+0x37/0x9a
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8058fe3e>] svc_process+0x38b/0x6e9
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8036db9d>] lockd+0x12d/0x186
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8036da70>] lockd+0x0/0x186
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8024becc>] kthread+0x47/0x73
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff80234852>] schedule_tail+0x27/0x5f
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8020cd19>] child_rip+0xa/0x11
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff804fa23c>] menu_reflect+0x0/0x6d
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8024be85>] kthread+0x0/0x73
<Dec/06 07:57 pm>[1742912.482643]  [<ffffffff8020cd0f>] child_rip+0x0/0x11
<Dec/06 07:57 pm>[1742968.257135] BUG: spinlock lockup on CPU#1, sh/4730, ffffffff80872a80
<Dec/06 07:57 pm>[1742968.264097] Pid: 4730, comm: sh Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:57 pm>[1742968.272380] Call Trace:
<Dec/06 07:57 pm>[1742968.275306]  [<ffffffff803a103f>] _raw_spin_lock+0xfb/0x122
<Dec/06 07:57 pm>[1742968.281398]  [<ffffffff805b789b>] lock_kernel+0x1d/0x2b
<Dec/06 07:57 pm>[1742968.287123]  [<ffffffff803e36ed>] tty_open+0x1d/0x414
<Dec/06 07:57 pm>[1742968.292743]  [<ffffffff802ae8ca>] chrdev_open+0x13c/0x18b
<Dec/06 07:57 pm>[1742968.298659]  [<ffffffff802ae78e>] chrdev_open+0x0/0x18b
<Dec/06 07:57 pm>[1742968.304397]  [<ffffffff802aa993>] __dentry_open+0x137/0x254
<Dec/06 07:57 pm>[1742968.310483]  [<ffffffff802b6829>] do_filp_open+0x439/0x7e1

<Dec/06 07:57 pm>[1742968.316497]  [<ffffffff805b795d>] error_exit+0x0/0x9a
<Dec/06 07:57 pm>[1742968.322084]  [<ffffffff805b7388>] _spin_unlock+0x17/0x20
<Dec/06 07:57 pm>[1742968.327944]  [<ffffffff802be3c9>] alloc_fd+0x103/0x112
<Dec/06 07:57 pm>[1742968.333627]  [<ffffffff802aa7b4>] do_sys_open+0x48/0xc5
<Dec/06 07:57 pm>[1742968.339383]  [<ffffffff8020bd2b>] system_call_fastpath+0x16/0x1b
<Dec/06 07:57 pm>[1742976.112791] BUG: soft lockup - CPU#6 stuck for 61s! [archttp64:8294]
<Dec/06 07:57 pm>[1742976.112793] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:57 pm>[1742976.112793] CPU 6:
<Dec/06 07:57 pm>[1742976.112793] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:57 pm>[1742976.112793] Pid: 8294, comm: archttp64 Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:57 pm>[1742976.112793] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 07:57 pm>[1742976.112793] RSP: 0018:ffff88063704bdf0  EFLAGS: 00000206
<Dec/06 07:57 pm>[1742976.112793] RAX: 0000000001a91236 RBX: 0000000001a91212 RCX: ffffffff802d6972
<Dec/06 07:57 pm>[1742976.112793] RDX: 00000000000edd63 RSI: ffff88063cc49960 RDI: 0000000000000001
<Dec/06 07:57 pm>[1742976.112793] RBP: 0000000001a91212 R08: ffff88063cc48380 R09: 0000000000000000
<Dec/06 07:57 pm>[1742976.112793] R10: 0000000000000000 R11: ffffffff8039d94e R12: 0000000000000000
<Dec/06 07:57 pm>[1742976.112793] R13: 0000000000000001 R14: 0000000000000000 R15: 0000000000000001
<Dec/06 07:57 pm>[1742976.112793] FS:  00000000006f8ca0(0063) GS:ffff88063e428c00(0000) knlGS:0000000000000000
<Dec/06 07:57 pm>[1742976.112793] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Dec/06 07:57 pm>[1742976.112793] CR2: 00007fab8afdb000 CR3: 000000063c013000 CR4: 00000000000006e0
<Dec/06 07:57 pm>[1742976.112793] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:57 pm>[1742976.112793] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

<Dec/06 07:57 pm>[1742976.112793] Call Trace:
<Dec/06 07:57 pm>[1742976.112793]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 07:57 pm>[1742976.112793]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 07:57 pm>[1742976.112793]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 07:57 pm>[1742976.112793]  [<ffffffff802d698f>] ? __posix_lock_file+0x6b/0x437
<Dec/06 07:57 pm>[1742976.112793]  [<ffffffff802d77f9>] ? fcntl_setlk+0x11d/0x26c
<Dec/06 07:57 pm>[1742976.112793]  [<ffffffff802b7650>] ? sys_fcntl+0x2ef/0x373
<Dec/06 07:57 pm>[1742976.112793]  [<ffffffff8020bd2b>] ? system_call_fastpath+0<Dec/06 07:57 pm>
                 <Dec/06 07:57 pm>[1742977.185983] BUG: soft lockup - CPU#4 stuck for 61s! [sh:4726]
<Dec/06 07:57 pm>[1742977.185986] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:57 pm>[1742977.185986] CPU 4:
<Dec/06 07:57 pm>[1742977.185986] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:57 pm>[1742977.185986] Pid: 4726, comm: sh Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:57 pm>[1742977.185986] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 07:57 pm>[1742977.185986] RSP: 0018:ffff8803d97bfcf0  EFLAGS: 00000202
<Dec/06 07:57 pm>[1742977.185986] RAX: 000000009b317fbe RBX: 000000007ae93d5d RCX: ffffffff807afb40
<Dec/06 07:57 pm>[1742977.185986] RDX: 00000000000edd63 RSI: ffff880456151900 RDI: 0000000000000001
<Dec/06 07:57 pm>[1742977.185986] RBP: 000000007ae93d5d R08: 0000000000000001 R09: 0000000000000000
<Dec/06 07:57 pm>[1742977.185986] R10: 0000000000000000 R11: ffffffff8039d94e R12: ffff8803d97b41ed
<Dec/06 07:57 pm>[1742977.185986] R13: ffff8805d750b005 R14: ffff88063d5c41ed R15: ffffffff802bac90
<Dec/06 07:57 pm>[1742977.185986] FS:  00007fedc60b26f0(0000) GS:ffff88063e428300(0000) knlGS:0000000000000000
<Dec/06 07:57 pm>[1742977.185986] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b

<Dec/06 07:57 pm>[1742977.185986] CR2: 00000000004931f8 CR3: 00000004f0345000 CR4: 00000000000006e0
<Dec/06 07:57 pm>[1742977.185986] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:57 pm>[1742977.185986] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 07:57 pm>[1742977.185986] Call Trace:
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff8039d963>] ? delay_tsc+0x15/0x42
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff803e36ed>] ? tty_open+0x1d/0x414
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff802ae8ca>] ? chrdev_open+0x13c/0x18b
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff802ae78e>] ? chrdev_open+0x0/0x18b
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff802aa993>] ? __dentry_open+0x137/0x254
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff802b6829>] ? do_filp_open+0x439/0x7e1
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff805b795d>] ? error_exit+0x0/0x9a
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 07:57 pm>[1742977.185986]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 07:57 pm>[1742986.602872] BUG: spinlock lockup on CPU#6, archttp64/8294, ffffffff80872a80
<Dec/06 07:57 pm>[1742986.610549] Pid: 8294, comm: archttp64 Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:57 pm>[1742986.619579] Call Trace:
<Dec/06 07:57 pm>[1742986.622516]  [<ffffffff803a103f>] _raw_spin_lock+0xfb/0x122
<Dec/06 07:57 pm>[1742986.628732]  [<ffffffff805b789b>] lock_kernel+0x1d/0x2b

<Dec/06 07:57 pm>[1742986.634573]  [<ffffffff802d698f>] __posix_lock_file+0x6b/0x437
<Dec/06 07:57 pm>[1742986.641047]  [<ffffffff802d77f9>] fcntl_setlk+0x11d/0x26c
<Dec/06 07:57 pm>[1742986.647037]  [<ffffffff802b7650>] sys_fcntl+0x2ef/0x373
<Dec/06 07:57 pm>[1742986.652872]  [<ffffffff8020bd2b>] system_call_fastpath+0x16/0x1b
<Dec/06 07:57 pm>[1742988.003249] BUG: spinlock lockup on CPU#4, sh/4726, ffffffff80872a80
<Dec/06 07:57 pm>[1742988.010244] Pid: 4726, comm: sh Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:57 pm>[1742988.018577] Call Trace:
<Dec/06 07:57 pm>[1742988.021542]  [<ffffffff803a103f>] _raw_spin_lock+0xfb/0x122
<Dec/06 07:57 pm>[1742988.027775]  [<ffffffff805b789b>] lock_kernel+0x1d/0x2b
<Dec/06 07:57 pm>[1742988.033606]  [<ffffffff803e36ed>] tty_open+0x1d/0x414
<Dec/06 07:57 pm>[1742988.039292]  [<ffffffff802ae8ca>] chrdev_open+0x13c/0x18b
<Dec/06 07:57 pm>[1742988.045298]  [<ffffffff802ae78e>] chrdev_open+0x0/0x18b
<Dec/06 07:57 pm>[1742988.051181]  [<ffffffff802aa993>] __dentry_open+0x137/0x254
<Dec/06 07:57 pm>[1742988.057412]  [<ffffffff802b6829>] do_filp_open+0x439/0x7e1
<Dec/06 07:57 pm>[1742988.063514]  [<ffffffff805b795d>] error_exit+0x0/0x9a
<Dec/06 07:57 pm>[1742988.069188]  [<ffffffff805b7388>] _spin_unlock+0x17/0x20
<Dec/06 07:57 pm>[1742988.075107]  [<ffffffff802be3c9>] alloc_fd+0x103/0x112
<Dec/06 07:57 pm>[1742988.080871]  [<ffffffff802aa7b4>] do_sys_open+0x48/0xc5
<Dec/06 07:57 pm>[1742988.086713]  [<ffffffff8020bd2b>] system_call_fastpath+0x16/0x1b
<Dec/06 07:57 pm>[1742993.292857] BUG: soft lockup - CPU#7 stuck for 61s! [ntpd:5084]
<Dec/06 07:57 pm>[1742993.292860] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:57 pm>[1742993.292860] CPU 7:

<Dec/06 07:57 pm>[1742993.292860] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:57 pm>[1742993.292860] Pid: 5084, comm: ntpd Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:57 pm>[1742993.292860] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 07:57 pm>[1742993.292860] RSP: 0018:ffff88059f905be0  EFLAGS: 00000202
<Dec/06 07:57 pm>[1742993.292860] RAX: 000000009b780f4e RBX: 000000009b780f2a RCX: ffff88033e008370
<Dec/06 07:57 pm>[1742993.292860] RDX: 00000000000edd6c RSI: ffff88063e000030 RDI: 0000000000000001
<Dec/06 07:57 pm>[1742993.292860] RBP: 000000009b780f2a R08: 0000000000000001 R09: 0000000000000000
<Dec/06 07:57 pm>[1742993.292860] R10: 0000000000000000 R11: ffffffff8039d94e R12: ffffffff802327e2
<Dec/06 07:57 pm>[1742993.292860] R13: ffff8802552c6a50 R14: 0000000000000000 R15: ffff8803224dcdc0
<Dec/06 07:57 pm>[1742993.292860] FS:  00007f838eb366f0(0000) GS:ffff88063e429000(0000) knlGS:0000000000000000
<Dec/06 07:57 pm>[1742993.292860] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Dec/06 07:57 pm>[1742993.292860] CR2: 000000002e4ace40 CR3: 00000004320b7000 CR4: 00000000000006e0
<Dec/06 07:57 pm>[1742993.292860] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:57 pm>[1742993.292860] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 07:57 pm>[1742993.292860] Call Trace:
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff802ed76d>] ? proc_lookup_de+0x18/0xdd
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff802e950e>] ? proc_root_lookup+0x12/0x30
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff802b30a5>] ? do_lookup+0xd9/0x1c3
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff802b4d63>] ? __link_path_walk+0x36c/0xcc9

<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff802b5706>] ? path_walk+0x46/0x8b
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff802b58d4>] ? do_path_lookup+0x10e/0x169
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff802b63af>] ? path_lookup_open+0x54/0x95
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff802b6492>] ? do_filp_open+0xa2/0x7e1
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 07:57 pm>[1742993.292860]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 07:58 pm>[1742999.923697] BUG: spinlock lockup on CPU#0, rpc.mountd/7611, ffffffff80872a80
<Dec/06 07:58 pm>[1742999.931520] Pid: 7611, comm: rpc.mountd Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:58 pm>[1742999.940518] Call Trace:
<Dec/06 07:58 pm>[1742999.943443]  [<ffffffff803a103f>] _raw_spin_lock+0xfb/0x122
<Dec/06 07:58 pm>[1742999.949586]  [<ffffffff805b789b>] lock_kernel+0x1d/0x2b
<Dec/06 07:58 pm>[1742999.955326]  [<ffffffff802d698f>] __posix_lock_file+0x6b/0x437
<Dec/06 07:58 pm>[1742999.961692]  [<ffffffff802d77f9>] fcntl_setlk+0x11d/0x26c
<Dec/06 07:58 pm>[1742999.967634]  [<ffffffff802b7650>] sys_fcntl+0x2ef/0x373
<Dec/06 07:58 pm>[1742999.973440]  [<ffffffff8020bd2b>] system_call_fastpath+0x16/0x1b
<Dec/06 07:58 pm>[1743003.853302] BUG: spinlock lockup on CPU#7, ntpd/5084, ffffffff80872a80
<Dec/06 07:58 pm>[1743003.860504] Pid: 5084, comm: ntpd Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:58 pm>[1743003.869045] Call Trace:
<Dec/06 07:58 pm>[1743003.872011]  [<ffffffff803a103f>] _raw_spin_lock+0xfb/0x122
<Dec/06 07:58 pm>[1743003.878197]  [<ffffffff805b789b>] lock_kernel+0x1d/0x2b

<Dec/06 07:58 pm>[1743003.884007]  [<ffffffff802ed76d>] proc_lookup_de+0x18/0xdd
<Dec/06 07:58 pm>[1743003.890130]  [<ffffffff802e950e>] proc_root_lookup+0x12/0x30
<Dec/06 07:58 pm>[1743003.896448]  [<ffffffff802b30a5>] do_lookup+0xd9/0x1c3
<Dec/06 07:58 pm>[1743003.902196]  [<ffffffff802b4d63>] __link_path_walk+0x36c/0xcc9
<Dec/06 07:58 pm>[1743003.908660]  [<ffffffff802b5706>] path_walk+0x46/0x8b
<Dec/06 07:58 pm>[1743003.914321]  [<ffffffff802b58d4>] do_path_lookup+0x10e/0x169
<Dec/06 07:58 pm>[1743003.920627]  [<ffffffff802b63af>] path_lookup_open+0x54/0x95
<Dec/06 07:58 pm>[1743003.931580]  [<ffffffff802b6492>] do_filp_open+0xa2/0x7e1
<Dec/06 07:58 pm>[1743003.937614]  [<ffffffff805b7388>] _spin_unlock+0x17/0x20
<Dec/06 07:58 pm>[1743003.943552]  [<ffffffff802be3c9>] alloc_fd+0x103/0x112
<Dec/06 07:58 pm>[1743003.949302]  [<ffffffff802aa7b4>] do_sys_open+0x48/0xc5
<Dec/06 07:58 pm>[1743003.955138]  [<ffffffff8020bd2b>] system_call_fastpath+0x16/0x1b
<Dec/06 07:58 pm>[1743033.022434] BUG: soft lockup - CPU#1 stuck for 61s! [sh:4730]
<Dec/06 07:58 pm>[1743033.022438] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:58 pm>[1743033.022438] CPU 1:
<Dec/06 07:58 pm>[1743033.022438] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:58 pm>[1743033.022438] Pid: 4730, comm: sh Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:58 pm>[1743033.022438] RIP: 0010:[<ffffffff80212063>]  [<ffffffff80212063>] native_read_tsc+0x6/0xc
<Dec/06 07:58 pm>[1743033.022438] RSP: 0018:ffff88011c96bcf0  EFLAGS: 00000206
<Dec/06 07:58 pm>[1743033.022438] RAX: 00000000cf3f5ae8 RBX: 00000000cf3f5ad6 RCX: ffffffff80932380
<Dec/06 07:58 pm>[1743033.022438] RDX: 000edd8200000000 RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 07:58 pm>[1743033.022438] RBP: 00000000cf3f5ad6 R08: ffffffff80239d5d R09: 0000000000000000

<Dec/06 07:58 pm>[1743033.022438] R10: 0000000000000000 R11: ffffffff8039d94e R12: 0000000040850c08
<Dec/06 07:58 pm>[1743033.022438] R13: 0000000040850c08 R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 07:58 pm>[1743033.022438] FS:  00007fb3ffd206f0(0000) GS:ffff88033e44d000(0000) knlGS:0000000000000000
<Dec/06 07:58 pm>[1743033.022438] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Dec/06 07:58 pm>[1743033.022438] CR2: 00000000004931f8 CR3: 0000000146cdd000 CR4: 00000000000006e0
<Dec/06 07:58 pm>[1743033.022438] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:58 pm>[1743033.022438] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 07:58 pm>[1743033.022438] Call Trace:
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff803e36ed>] ? tty_open+0x1d/0x414
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff802ae8ca>] ? chrdev_open+0x13c/0x18b
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff802ae78e>] ? chrdev_open+0x0/0x18b
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff802aa993>] ? __dentry_open+0x137/0x254
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff802b6829>] ? do_filp_open+0x439/0x7e1
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff805b795d>] ? error_exit+0x0/0x9a
                 <Dec/06 07:58 pm>[1743033.022438]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 07:58 pm>[1743033.022438]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 07:58 pm>[1743051.276115] BUG: soft lockup - CPU#6 stuck for 61s! [archttp64:8294]

<Dec/06 07:58 pm>[1743051.276118] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:58 pm>[1743051.276118] CPU 6:
<Dec/06 07:58 pm>[1743051.276118] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:58 pm>[1743051.276118] Pid: 8294, comm: archttp64 Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:58 pm>[1743051.276118] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 07:58 pm>[1743051.276118] RSP: 0018:ffff88063704bdf0  EFLAGS: 00000202
<Dec/06 07:58 pm>[1743051.276118] RAX: 0000000002a60afa RBX: 0000000002a60ad6 RCX: ffffffff80932380
<Dec/06 07:58 pm>[1743051.276118] RDX: 00000000000edd8d RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 07:58 pm>[1743051.276118] RBP: 0000000002a60ad6 R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 07:58 pm>[1743051.276118] R10: 0000000000000000 R11: ffffffff8039d94e R12: 000000007c08d1fa
<Dec/06 07:58 pm>[1743051.276118] R13: 000000007c08d1fa R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 07:58 pm>[1743051.276118] FS:  00000000006f8ca0(0063) GS:ffff88063e428c00(0000) knlGS:0000000000000000
<Dec/06 07:58 pm>[1743051.276118] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Dec/06 07:58 pm>[1743051.276118] CR2: 00007fab8afdb000 CR3: 000000063c013000 CR4: 00000000000006e0
<Dec/06 07:58 pm>[1743051.276118] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:58 pm>[1743051.276118] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 07:58 pm>[1743051.276118] Call Trace:
<Dec/06 07:58 pm>[1743051.276118]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 07:58 pm>[1743051.276118]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 07:58 pm>[1743051.276118]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 07:58 pm>[1743051.276118]  [<ffffffff802d698f>] ? __posix_lock_file+0x6b/0x437
<Dec/06 07:58 pm>[1743051.276118]  [<ffffffff802d77f9>] ? fcntl_setlk+0x11d/0x26c

<Dec/06 07:58 pm>[1743051.276118]  [<ffffffff802b7650>] ? sys_fcntl+0x2ef/0x373
                 <Dec/06 07:58 pm>[1743051.276118]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 07:58 pm>[1743053.422641] BUG: soft lockup - CPU#4 stuck for 61s! [sh:4726]
<Dec/06 07:58 pm>[1743053.422646] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last un<Dec/06 07:58 pm>
                 <Dec/06 07:58 pm>[1743053.422646] CPU 4:
<Dec/06 07:58 pm>[1743053.422646] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:58 pm>[1743053.422646] Pid: 4726, comm: sh Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:58 pm>[1743053.422646] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 07:58 pm>[1743053.422646] RSP: 0018:ffff8803d97bfcf0  EFLAGS: 00000202
<Dec/06 07:58 pm>[1743053.422646] RAX: 0000000035bbfe3e RBX: 0000000035bbfe1a RCX: ffffffff80932380
<Dec/06 07:58 pm>[1743053.422646] RDX: 00000000000edd8e RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 07:58 pm>[1743053.422646] RBP: 0000000035bbfe1a R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 07:58 pm>[1743053.422646] R10: 0000000000000000 R11: ffffffff8039d94e R12: 0000000049889ca8
<Dec/06 07:58 pm>[1743053.422646] R13: 0000000049889ca8 R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 07:58 pm>[1743053.422646] FS:  00007fedc60b26f0(0000) GS:ffff88063e428300(0000) knlGS:0000000000000000
<Dec/06 07:58 pm>[1743053.422646] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Dec/06 07:58 pm>[1743053.422646] CR2: 00000000004931f8 CR3: 00000004f0345000 CR4: 00000000000006e0
<Dec/06 07:58 pm>[1743053.422646] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:58 pm>[1743053.422646] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 07:58 pm>[1743053.422646] Call Trace:
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122

<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff803e36ed>] ? tty_open+0x1d/0x414
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff802ae8ca>] ? chrdev_open+0x13c/0x18b
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff802ae78e>] ? chrdev_open+0x0/0x18b
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff802aa993>] ? __dentry_open+0x137/0x254
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff802b6829>] ? do_filp_open+0x439/0x7e1
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff805b795d>] ? error_exit+0x0/0x9a
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 07:58 pm>[1743053.422646]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 07:59 pm>[1743065.235696] BUG: soft lockup - CPU#0 stuck for 61s! [rpc.mountd:7611]
<Dec/06 07:59 pm>[1743065.235699] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:59 pm>[1743065.235699] CPU 0:
<Dec/06 07:59 pm>[1743065.235699] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:59 pm>[1743065.235699] Pid: 7611, comm: rpc.mountd Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:59 pm>[1743065.235699] RIP: 0010:[<ffffffff8039d963>]  [<ffffffff8039d963>] delay_tsc+0x15/0x42
<Dec/06 07:59 pm>[1743065.235699] RSP: 0018:ffff88025ef9fdf8  EFLAGS: 00000202
<Dec/06 07:59 pm>[1743065.235699] RAX: 000edd94cfbaaace RBX: 0000000014328127 RCX: ffffffff80932380
<Dec/06 07:59 pm>[1743065.235699] RDX: 000edd9400000000 RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 07:59 pm>[1743065.235699] RBP: 0000000014328127 R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 07:59 pm>[1743065.235699] R10: 0000000000000000 R11: ffffffff8039d94e R12: 00000000ee12c1ae

<Dec/06 07:59 pm>[1743065.235699] R13: 00000000ee12c1ae R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 07:59 pm>[1743065.235699] FS:  00007fa53d3b2740(0000) GS:ffffffff80872d80(0000) knlGS:0000000000000000
<Dec/06 07:59 pm>[1743065.235699] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Dec/06 07:59 pm>[1743065.235699] CR2: 00007fa53d3cb000 CR3: 0000000303295000 CR4: 00000000000006e0
<Dec/06 07:59 pm>[1743065.235699] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:59 pm>[1743065.235699] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 07:59 pm>[1743065.235699] Call Trace:
<Dec/06 07:59 pm>[1743065.235699]  [<ffffffff8039d963>] ? delay_tsc+0x15/0x42
<Dec/06 07:59 pm>[1743065.235699]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 07:59 pm>[1743065.235699]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 07:59 pm>[1743065.235699]  [<ffffffff802d698f>] ? __posix_lock_file+0x6b/0x437
<Dec/06 07:59 pm>[1743065.235699]  [<ffffffff802d77f9>] ? fcntl_setlk+0x11d/0x26c
<Dec/06 07:59 pm>[1743065.235699]  [<ffffffff802b7650>] ? sys_fcntl+0x2ef/0x373
<Dec/06 07:59 pm>[1743065.235699]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 07:59 pm>[1743068.456182] BUG: soft lockup - CPU#7 stuck for 61s! [ntpd:5084]
<Dec/06 07:59 pm>[1743068.456186] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:59 pm>[1743068.456186] CPU 7:
<Dec/06 07:59 pm>[1743068.456186] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:59 pm>[1743068.456186] Pid: 5084, comm: ntpd Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:59 pm>[1743068.456186] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 07:59 pm>[1743068.456186] RSP: 0018:ffff88059f905be0  EFLAGS: 00000206
<Dec/06 07:59 pm>[1743068.456186] RAX: 000000009c75085a RBX: 000000009c750836 RCX: ffffffff80932380

<Dec/06 07:59 pm>[1743068.456186] RDX: 00000000000edd96 RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 07:59 pm>[1743068.456186] RBP: 000000009c750836 R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 07:59 pm>[1743068.456186] R10: 0000000000000000 R11: ffffffff8039d94e R12: 00000000277c3c52
<Dec/06 07:59 pm>[1743068.456186] R13: 00000000277c3c52 R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 07:59 pm>[1743068.456186] FS:  00007f838eb366f0(0000) GS:ffff88063e429000(0000) knlGS:0000000000000000
<Dec/06 07:59 pm>[1743068.456186] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Dec/06 07:59 pm>[1743068.456186] CR2: 000000002e4ace40 CR3: 00000004320b7000 CR4: 00000000000006e0
<Dec/06 07:59 pm>[1743068.456186] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:59 pm>[1743068.456186] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 07:59 pm>[1743068.456186] Call Trace:
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff802ed76d>] ? proc_lookup_de+0x18/0xdd
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff802e950e>] ? proc_root_lookup+0x12/0x30
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff802b30a5>] ? do_lookup+0xd9/0x1c3
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff802b4d63>] ? __link_path_walk+0x36c/0xcc9
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff802b5706>] ? path_walk+0x46/0x8b
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff802b58d4>] ? do_path_lookup+0x10e/0x169
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff802b63af>] ? path_lookup_open+0x54/0x95
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff802b6492>] ? do_filp_open+0xa2/0x7e1
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20

<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 07:59 pm>[1743068.456186]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 07:59 pm>[1743098.519095] BUG: soft lockup - CPU#1 stuck for 61s! [sh:4730]
<Dec/06 07:59 pm>[1743098.519098] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:59 pm>[1743098.519098] CPU 1:
<Dec/06 07:59 pm>[1743098.519098] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:59 pm>[1743098.519098] Pid: 4730, comm: sh Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:59 pm>[1743098.519098] RIP: 0010:[<ffffffff8039d98b>]  [<ffffffff8039d98b>] delay_tsc+0x3d/0x42
<Dec/06 07:59 pm>[1743098.519098] RSP: 0018:ffff88011c96bcf8  EFLAGS: 00000206
<Dec/06 07:59 pm>[1743098.519098] RAX: 0000000000000012 RBX: 00000000694e81f9 RCX: ffffffff80932380
<Dec/06 07:59 pm>[1743098.519098] RDX: 000edda700000000 RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 07:59 pm>[1743098.519098] RBP: 00000000694e81f9 R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 07:59 pm>[1743098.519098] R10: 0000000000000000 R11: ffffffff8039d94e R12: 0000000040850c08
<Dec/06 07:59 pm>[1743098.519098] R13: 0000000040850c08 R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 07:59 pm>[1743098.519098] FS:  00007fb3ffd206f0(0000) GS:ffff88033e44d000(0000) knlGS:0000000000000000
<Dec/06 07:59 pm>[1743098.519098] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Dec/06 07:59 pm>[1743098.519098] CR2: 00000000004931f8 CR3: 0000000146cdd000 CR4: 00000000000006e0
<Dec/06 07:59 pm>[1743098.519098] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:59 pm>[1743098.519098] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 07:59 pm>[1743098.519098] Call Trace:
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42

<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff803e36ed>] ? tty_open+0x1d/0x414
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff802ae8ca>] ? chrdev_open+0x13c/0x18b
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff802ae78e>] ? chrdev_open+0x0/0x18b
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff802aa993>] ? __dentry_open+0x137/0x254
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff802b6829>] ? do_filp_open+0x439/0x7e1
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff805b795d>] ? error_exit+0x0/0x9a
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 07:59 pm>[1743098.519098]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 07:59 pm>[1743116.772774] BUG: soft lockup - CPU#6 stuck for 61s! [archttp64:8294]
<Dec/06 07:59 pm>[1743116.772778] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:59 pm>[1743116.772778] CPU 6:
<Dec/06 07:59 pm>[1743116.772778] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 07:59 pm>[1743116.772778] Pid: 8294, comm: archttp64 Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 07:59 pm>[1743116.772778] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 07:59 pm>[1743116.772778] RSP: 0018:ffff88063704bdf0  EFLAGS: 00000206
<Dec/06 07:59 pm>[1743116.772778] RAX: 000000009cb531b4 RBX: 0000000055220a3f RCX: ffffffff80932380
<Dec/06 07:59 pm>[1743116.772778] RDX: 00000000000eddb1 RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 07:59 pm>[1743116.772778] RBP: 0000000055220a3f R08: ffffffff80239d5d R09: 0000000000000000

<Dec/06 07:59 pm>[1743116.772778] R10: 0000000000000000 R11: ffffffff8039d94e R12: 000000007c08d1fa
<Dec/06 07:59 pm>[1743116.772778] R13: 000000007c08d1fa R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 07:59 pm>[1743116.772778] FS:  00000000006f8ca0(0063) GS:ffff88063e428c00(0000) knlGS:0000000000000000
<Dec/06 07:59 pm>[1743116.772778] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Dec/06 07:59 pm>[1743116.772778] CR2: 00007fab8afdb000 CR3: 000000063c013000 CR4: 00000000000006e0
<Dec/06 07:59 pm>[1743116.772778] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 07:59 pm>[1743116.772778] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 07:59 pm>[1743116.772778] Call Trace:
<Dec/06 07:59 pm>[1743116.772778]  [<ffffffff8039d963>] ? delay_tsc+0x15/0x42
<Dec/06 07:59 pm>[1743116.772778]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 07:59 pm>[1743116.772778]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 07:59 pm>[1743116.772778]  [<ffffffff802d698f>] ? __posix_lock_file+0x6b/0x437
<Dec/06 07:59 pm>[1743116.772778]  [<ffffffff802d77f9>] ? fcntl_setlk+0x11d/0x26c
<Dec/06 07:59 pm>[1743116.772778]  [<ffffffff802b7650>] ? sys_fcntl+0x2ef/0x373
<Dec/06 07:59 pm>[1743116.772778]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 08:00 pm>[1743118.922635] BUG: soft lockup - CPU#4 stuck for 61s! [sh:4726]
<Dec/06 08:00 pm>[1743118.922639] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:00 pm>[1743118.922639] CPU 4:
<Dec/06 08:00 pm>[1743118.922639] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:00 pm>[1743118.922639] Pid: 4726, comm: sh Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 08:00 pm>[1743118.922639] RIP: 0010:[<ffffffff80212063>]  [<ffffffff80212063>] native_read_tsc+0x6/0xc
<Dec/06 08:00 pm>[1743118.922639] RSP: 0018:ffff8803d97bfcf0  EFLAGS: 00000206

<Dec/06 08:00 pm>[1743118.922639] RAX: 00000000d0453948 RBX: 00000000d0453924 RCX: ffffffff80932380
<Dec/06 08:00 pm>[1743118.922639] RDX: 000eddb200000000 RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 08:00 pm>[1743118.922639] RBP: 00000000d0453924 R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 08:00 pm>[1743118.922639] R10: 0000000000000000 R11: ffffffff8039d94e R12: 0000000049889ca8
<Dec/06 08:00 pm>[1743118.922639] R13: 0000000049889ca8 R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 08:00 pm>[1743118.922639] FS:  00007fedc60b26f0(0000) GS:ffff88063e428300(0000) knlGS:0000000000000000
<Dec/06 08:00 pm>[1743118.922639] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Dec/06 08:00 pm>[1743118.922639] CR2: 00000000004931f8 CR3: 00000004f0345000 CR4: 00000000000006e0
<Dec/06 08:00 pm>[1743118.922639] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 08:00 pm>[1743118.922639] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 08:00 pm>[1743118.922639] Call Trace:
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff803e36ed>] ? tty_open+0x1d/0x414
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff802ae8ca>] ? chrdev_open+0x13c/0x18b
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff802ae78e>] ? chrdev_open+0x0/0x18b
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff802aa993>] ? __dentry_open+0x137/0x254
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff802b6829>] ? do_filp_open+0x439/0x7e1
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff805b795d>] ? error_exit+0x0/0x9a
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112

<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 08:00 pm>[1743118.922639]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 08:00 pm>[1743130.732357] BUG: soft lockup - CPU#0 stuck for 61s! [rpc.mountd:7611]
<Dec/06 08:00 pm>[1743130.732359] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:00 pm>[1743130.732359] CPU 0:
                 <Dec/06 08:00 pm>[1743130.732359] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:00 pm>[1743130.732359] Pid: 7611, comm: rpc.mountd Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 08:00 pm>[1743130.732359] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 08:00 pm>[1743130.732359] RSP: 0018:ffff88025ef9fdf0  EFLAGS: 00000206
<Dec/06 08:00 pm>[1743130.732359] RAX: 0000000069c9d2e1 RBX: 0000000069c9d2cf RCX: ffffffff80932380
<Dec/06 08:00 pm>[1743130.732359] RDX: 00000000000eddb9 RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 08:00 pm>[1743130.732359] RBP: 0000000069c9d2cf R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 08:00 pm>[1743130.732359] R10: 0000000000000000 R11: ffffffff8039d94e R12: 00000000ee12c1ae
<Dec/06 08:00 pm>[1743130.732359] R13: 00000000ee12c1ae R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 08:00 pm>[1743130.732359] FS:  00007fa53d3b2740(0000) GS:ffffffff80872d80(0000) knlGS:0000000000000000
<Dec/06 08:00 pm>[1743130.732359] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Dec/06 08:00 pm>[1743130.732359] CR2: 00007fa53d3cb000 CR3: 0000000303295000 CR4: 00000000000006e0
<Dec/06 08:00 pm>[1743130.732359] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 08:00 pm>[1743130.732359] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 08:00 pm>[1743130.732359] Call Trace:
<Dec/06 08:00 pm>[1743130.732359]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 08:00 pm>[1743130.732359]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122

<Dec/06 08:00 pm>[1743130.732359]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 08:00 pm>[1743130.732359]  [<ffffffff802d698f>] ? __posix_lock_file+0x6b/0x437
<Dec/06 08:00 pm>[1743130.732359]  [<ffffffff802d77f9>] ? fcntl_setlk+0x11d/0x26c
<Dec/06 08:00 pm>[1743130.732359]  [<ffffffff802b7650>] ? sys_fcntl+0x2ef/0x373
<Dec/06 08:00 pm>[1743130.732359]  [<ffffffff8020bd2b>] ? system_call_fastpath+0<Dec/06 08:00 pm>
                 <Dec/06 08:00 pm>[1743133.952842] BUG: soft lockup - CPU#7 stuck for 61s! [ntpd:5084]
<Dec/06 08:00 pm>[1743133.952846] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:00 pm>[1743133.952846] CPU 7:
<Dec/06 08:00 pm>[1743133.952846] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:00 pm>[1743133.952846] Pid: 5084, comm: ntpd Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 08:00 pm>[1743133.952846] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 08:00 pm>[1743133.952846] RSP: 0018:ffff88059f905be0  EFLAGS: 00000202
<Dec/06 08:00 pm>[1743133.952846] RAX: 0000000036842f62 RBX: 0000000036842f3e RCX: ffffffff80932380
<Dec/06 08:00 pm>[1743133.952846] RDX: 00000000000eddbb RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 08:00 pm>[1743133.952846] RBP: 0000000036842f3e R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 08:00 pm>[1743133.952846] R10: 0000000000000000 R11: ffffffff8039d94e R12: 00000000277c3c52
<Dec/06 08:00 pm>[1743133.952846] R13: 00000000277c3c52 R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 08:00 pm>[1743133.952846] FS:  00007f838eb366f0(0000) GS:ffff88063e429000(0000) knlGS:0000000000000000
<Dec/06 08:00 pm>[1743133.952846] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Dec/06 08:00 pm>[1743133.952846] CR2: 000000002e4ace40 CR3: 00000004320b7000 CR4: 00000000000006e0
<Dec/06 08:00 pm>[1743133.952846] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 08:00 pm>[1743133.952846] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400

<Dec/06 08:00 pm>[1743133.952846] Call Trace:
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff802ed76d>] ? proc_lookup_de+0x18/0xdd
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff802e950e>] ? proc_root_lookup+0x12/0x30
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff802b30a5>] ? do_lookup+0xd9/0x1c3
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff802b4d63>] ? __link_path_walk+0x36c/0xcc9
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff802b5706>] ? path_walk+0x46/0x8b
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff802b58d4>] ? do_path_lookup+0x10e/0x169
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff802b63af>] ? path_lookup_open+0x54/0x95
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff802b6492>] ? do_filp_open+0xa2/0x7e1
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 08:00 pm>[1743133.952846]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 08:00 pm>[1743164.019088] BUG: soft lockup - CPU#1 stuck for 61s! [sh:4730]
<Dec/06 08:00 pm>[1743164.019091] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:00 pm>[1743164.019091] CPU 1:
<Dec/06 08:00 pm>[1743164.019091] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:00 pm>[1743164.019091] Pid: 4730, comm: sh Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 08:00 pm>[1743164.019091] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc

<Dec/06 08:00 pm>[1743164.019091] RSP: 0018:ffff88011c96bcf0  EFLAGS: 00000202
<Dec/06 08:00 pm>[1743164.019091] RAX: 0000000003d7bc6a RBX: 0000000003d7bc58 RCX: ffffffff80932380
<Dec/06 08:00 pm>[1743164.019091] RDX: 00000000000eddcc RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 08:00 pm>[1743164.019091] RBP: 0000000003d7bc58 R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 08:00 pm>[1743164.019091] R10: 0000000000000000 R11: ffffffff8039d94e R12: 0000000040850c08
<Dec/06 08:00 pm>[1743164.019091] R13: 0000000040850c08 R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 08:00 pm>[1743164.019091] FS:  00007fb3ffd206f0(0000) GS:ffff88033e44d000(0000) knlGS:0000000000000000
<Dec/06 08:00 pm>[1743164.019091] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Dec/06 08:00 pm>[1743164.019091] CR2: 00000000004931f8 CR3: 0000000146cdd000 CR4: 00000000000006e0
<Dec/06 08:00 pm>[1743164.019091] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 08:00 pm>[1743164.019091] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 08:00 pm>[1743164.019091] Call Trace:
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff803e36ed>] ? tty_open+0x1d/0x414
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff802ae8ca>] ? chrdev_open+0x13c/0x18b
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff802ae78e>] ? chrdev_open+0x0/0x18b
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff802aa993>] ? __dentry_open+0x137/0x254
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff802b6829>] ? do_filp_open+0x439/0x7e1
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff805b795d>] ? error_exit+0x0/0x9a
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20

<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 08:00 pm>[1743164.019091]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 08:01 pm>[1743182.272768] BUG: soft lockup - CPU#6 stuck for 61s! [archttp64:8294]
<Dec/06 08:01 pm>[1743182.272773] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:01 pm>[1743182.272773] CPU 6:
<Dec/06 08:01 pm>[1743182.272773] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:01 pm>[1743182.272773] Pid: 8294, comm: archttp64 Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 08:01 pm>[1743182.272773] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 08:01 pm>[1743182.272773] RSP: 0018:ffff88063704bdf0  EFLAGS: 00000202
<Dec/06 08:01 pm>[1743182.272773] RAX: 00000000373e6dc6 RBX: 00000000373e6da2 RCX: ffffffff80932380
<Dec/06 08:01 pm>[1743182.272773] RDX: 00000000000eddd6 RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 08:01 pm>[1743182.272773] RBP: 00000000373e6da2 R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 08:01 pm>[1743182.272773] R10: 0000000000000000 R11: ffffffff8039d94e R12: 000000007c08d1fa
<Dec/06 08:01 pm>[1743182.272773] R13: 000000007c08d1fa R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 08:01 pm>[1743182.272773] FS:  00000000006f8ca0(0063) GS:ffff88063e428c00(0000) knlGS:0000000000000000
<Dec/06 08:01 pm>[1743182.272773] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Dec/06 08:01 pm>[1743182.272773] CR2: 00007fab8afdb000 CR3: 000000063c013000 CR4: 00000000000006e0
<Dec/06 08:01 pm>[1743182.272773] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 08:01 pm>[1743182.272773] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 08:01 pm>[1743182.272773] Call Trace:
<Dec/06 08:01 pm>[1743182.272773]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42

<Dec/06 08:01 pm>[1743182.272773]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 08:01 pm>[1743182.272773]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 08:01 pm>[1743182.272773]  [<ffffffff802d698f>] ? __posix_lock_file+0x6b/0x437
<Dec/06 08:01 pm>[1743182.272773]  [<ffffffff802d77f9>] ? fcntl_setlk+0x11d/0x26c
<Dec/06 08:01 pm>[1743182.272773]  [<ffffffff802b7650>] ? sys_fcntl+0x2ef/0x373
<Dec/06 08:01 pm>[1743182.272773]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 08:01 pm>[1743184.419296] BUG: soft lockup - CPU#4 stuck for 61s! [sh:4726]
<Dec/06 08:01 pm>[1743184.419300] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:01 pm>[1743184.419300] CPU 4:
<Dec/06 08:01 pm>[1743184.419300] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:01 pm>[1743184.419300] Pid: 4726, comm: sh Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 08:01 pm>[1743184.419300] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 08:01 pm>[1743184.419300] RSP: 0018:ffff8803d97bfcf0  EFLAGS: 00000202
<Dec/06 08:01 pm>[1743184.419300] RAX: 000000006a54609e RBX: 00000000439484b3 RCX: ffffffff80932380
<Dec/06 08:01 pm>[1743184.419300] RDX: 00000000000eddd7 RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 08:01 pm>[1743184.419300] RBP: 00000000439484b3 R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 08:01 pm>[1743184.419300] R10: 0000000000000000 R11: ffffffff8039d94e R12: 0000000049889ca8
<Dec/06 08:01 pm>[1743184.419300] R13: 0000000049889ca8 R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 08:01 pm>[1743184.419300] FS:  00007fedc60b26f0(0000) GS:ffff88063e428300(0000) knlGS:0000000000000000
<Dec/06 08:01 pm>[1743184.419300] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
<Dec/06 08:01 pm>[1743184.419300] CR2: 00000000004931f8 CR3: 00000004f0345000 CR4: 00000000000006e0
<Dec/06 08:01 pm>[1743184.419300] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000

<Dec/06 08:01 pm>[1743184.419300] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 08:01 pm>[1743184.419300] Call Trace:
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff8039d963>] ? delay_tsc+0x15/0x42
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff803e36ed>] ? tty_open+0x1d/0x414
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff802ae8ca>] ? chrdev_open+0x13c/0x18b
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff802ae78e>] ? chrdev_open+0x0/0x18b
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff802aa993>] ? __dentry_open+0x137/0x254
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff802b6829>] ? do_filp_open+0x439/0x7e1
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff805b795d>] ? error_exit+0x0/0x9a
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 08:01 pm>[1743184.419300]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 08:01 pm>[1743196.229015] BUG: soft lockup - CPU#0 stuck for 61s! [rpc.mountd:7611]
<Dec/06 08:01 pm>[1743196.229019] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:01 pm>[1743196.229019] CPU 0:
<Dec/06 08:01 pm>[1743196.229019] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:01 pm>[1743196.229019] Pid: 7611, comm: rpc.mountd Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 08:01 pm>[1743196.229019] RIP: 0010:[<ffffffff8021205f>]  [<ffffffff8021205f>] native_read_tsc+0x2/0xc
<Dec/06 08:01 pm>[1743196.229019] RSP: 0018:ffff88025ef9fdf0  EFLAGS: 00000206

<Dec/06 08:01 pm>[1743196.229019] RAX: 0000000003d8f90b RBX: 0000000003d8f8f9 RCX: ffffffff80932380
<Dec/06 08:01 pm>[1743196.229019] RDX: 00000000000eddde RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 08:01 pm>[1743196.229019] RBP: 0000000003d8f8f9 R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 08:01 pm>[1743196.229019] R10: 0000000000000000 R11: ffffffff8039d94e R12: 00000000ee12c1ae
<Dec/06 08:01 pm>[1743196.229019] R13: 00000000ee12c1ae R14: ffffffff8020e25d R15: ffffffff80794266
<Dec/06 08:01 pm>[1743196.229019] FS:  00007fa53d3b2740(0000) GS:ffffffff80872d80(0000) knlGS:0000000000000000
<Dec/06 08:01 pm>[1743196.229019] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Dec/06 08:01 pm>[1743196.229019] CR2: 00007fa53d3cb000 CR3: 0000000303295000 CR4: 00000000000006e0
<Dec/06 08:01 pm>[1743196.229019] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 08:01 pm>[1743196.229019] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 08:01 pm>[1743196.229019] Call Trace:
<Dec/06 08:01 pm>[1743196.229019]  [<ffffffff8039d96a>] ? delay_tsc+0x1c/0x42
<Dec/06 08:01 pm>[1743196.229019]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 08:01 pm>[1743196.229019]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 08:01 pm>[1743196.229019]  [<ffffffff802d698f>] ? __posix_lock_file+0x6b/0x437
<Dec/06 08:01 pm>[1743196.229019]  [<ffffffff802d77f9>] ? fcntl_setlk+0x11d/0x26c
<Dec/06 08:01 pm>[1743196.229019]  [<ffffffff802b7650>] ? sys_fcntl+0x2ef/0x373
<Dec/06 08:01 pm>[1743196.229019]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b
<Dec/06 08:01 pm>[1743199.452836] BUG: soft lockup - CPU#7 stuck for 61s! [ntpd:5084]
<Dec/06 08:01 pm>[1743199.452840] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]
<Dec/06 08:01 pm>[1743199.452840] CPU 7:
<Dec/06 08:01 pm>[1743199.452840] Modules linked in: hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan [last unloaded: w83793]

<Dec/06 08:01 pm>[1743199.452840] Pid: 5084, comm: ntpd Tainted: G      D W  2.6.28-rc7-00105-gfeaf384 #4
<Dec/06 08:01 pm>[1743199.452840] RIP: 0010:[<ffffffff803a0fe7>]  [<ffffffff803a0fe7>] _raw_spin_lock+0xa3/0x122
<Dec/06 08:01 pm>[1743199.452840] RSP: 0018:ffff88059f905c08  EFLAGS: 00000297
<Dec/06 08:01 pm>[1743199.452840] RAX: 000000000000a1a0 RBX: 0000000041ed981e RCX: ffffffff80932380
<Dec/06 08:01 pm>[1743199.452840] RDX: 000000000000a2a0 RSI: 0000000001062560 RDI: 0000000000000001
<Dec/06 08:01 pm>[1743199.452840] RBP: 0000000041ed981e R08: ffffffff80239d5d R09: 0000000000000000
<Dec/06 08:01 pm>[1743199.452840] R10: 0000000000000000 R11: ffffffff8039d94e R12: 0000000000000000
<Dec/06 08:01 pm>[1743199.452840] R13: ffffffff8039d94e R14: 00000000d0935724 R15: 00000000d0935724
<Dec/06 08:01 pm>[1743199.452840] FS:  00007f838eb366f0(0000) GS:ffff88063e429000(0000) knlGS:0000000000000000
<Dec/06 08:01 pm>[1743199.452840] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<Dec/06 08:01 pm>[1743199.452840] CR2: 000000002e4ace40 CR3: 00000004320b7000 CR4: 00000000000006e0
<Dec/06 08:01 pm>[1743199.452840] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/06 08:01 pm>[1743199.452840] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/06 08:01 pm>[1743199.452840] Call Trace:
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff803a0ffe>] ? _raw_spin_lock+0xba/0x122
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff805b789b>] ? lock_kernel+0x1d/0x2b
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff802ed76d>] ? proc_lookup_de+0x18/0xdd
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff802e950e>] ? proc_root_lookup+0x12/0x30
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff802b30a5>] ? do_lookup+0xd9/0x1c3
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff802b4d63>] ? __link_path_walk+0x36c/0xcc9
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff802b5706>] ? path_walk+0x46/0x8b
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff802b58d4>] ? do_path_lookup+0x10e/0x169

<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff802b63af>] ? path_lookup_open+0x54/0x95
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff802b6492>] ? do_filp_open+0xa2/0x7e1
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff805b7388>] ? _spin_unlock+0x17/0x20
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff802be3c9>] ? alloc_fd+0x103/0x112
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff802aa7b4>] ? do_sys_open+0x48/0xc5
<Dec/06 08:01 pm>[1743199.452840]  [<ffffffff8020bd2b>] ? system_call_fastpath+0x16/0x1b

Last modified 14 years ago Last modified on Dec 6, 2012, 8:19:31 PM

Note: See TracWiki for help on using the wiki.

Download in other formats:

Plain Text