Ippc28-crash-20121224 – Pan-STARRS IPP

wiki:Ippc28-crash-20121224

Context Navigation

from console -- power cycle stalled a while on starting local but seems to be back up

<Dec/24 01:40 am>ippc28 login: [1507747.909968] general protection fault: 0000 [#1] SMP 
<Dec/24 01:40 am>[1507747.912564] last sysfs file: /sys/class/i2c-adapter/i2c-0/0-002f/temp6_alarm
<Dec/24 01:40 am>[1507747.912564] CPU 0 
<Dec/24 01:40 am>[1507747.912564] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
<Dec/24 01:40 am>[1507747.912564] Pid: 1671, comm: rpciod/0 Not tainted 2.6.28-rc7-00105-gfeaf384 #4
<Dec/24 01:40 am>[1507747.912564] RIP: 0010:[<ffffffff802488fd>]  [<ffffffff802488fd>] run_workqueue+0x77/0x1ed
<Dec/24 01:40 am>[1507747.912564] RSP: 0018:ffff88063b293e80  EFLAGS: 00010087
<Dec/24 01:40 am>[1507747.912564] RAX: 1aa494a4d74ac649 RBX: ffff88013f277580 RCX: 0000000000000000
<Dec/24 01:40 am>[1507747.912564] RDX: ffff8809f4e1b4f8 RSI: ffff88013f2775b0 RDI: ffff88063b233000
<Dec/24 01:40 am>[1507747.912564] RBP: ffff88063b233000 R08: 0000000000000001 R09: 0000000000000001
<Dec/24 01:40 am>[1507747.912564] R10: 0000000000000000 R11: 0000000000000000 R12: ffff88013f277578
<Dec/24 01:40 am>[1507747.912564] R13: cc4a7f6d487a4100 R14: ffffffff808b85a0 R15: 0000000000000000
<Dec/24 01:40 am>[1507747.912564] FS:  0000000000000000(0000) GS:ffffffff80872d80(0000) knlGS:0000000000000000

<Dec/24 01:40 am>[1507747.912564] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
<Dec/24 01:40 am>[1507747.912564] CR2: 0000000000a1c008 CR3: 0000000638454000 CR4: 00000000000006e0
<Dec/24 01:40 am>[1507747.912564] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/24 01:40 am>[1507747.912564] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/24 01:40 am>[1507747.912564] Process rpciod/0 (pid: 1671, threadinfo ffff88063b292000, task ffff88063b219e60)
<Dec/24 01:40 am>[1507747.912564] Stack:
<Dec/24 01:40 am>[1507747.912564]  0000000000000001 0000000000000046 5f185ad49f1d7d91 95c4cf635cb05640
<Dec/24 01:40 am>[1507747.912564]  a3d560adfe0c5e41 ffff88063b233000 ffff88063cc6dbf0 ffff88063b233000
<Dec/24 01:40 am>[1507748.069655]  ffff88063b233040 ffff88063cc6dbf0 ffffffff808b5d60 ffffffff80249511
<Dec/24 01:40 am>[1507748.069655] Call Trace:
<Dec/24 01:40 am>[1507748.069655]  [<ffffffff80249511>] ? worker_thread+0xd8/0xe3
<Dec/24 01:40 am>[1507748.069655]  [<ffffffff8024bfda>] ? autoremove_wake_function+0x0/0x2e
<Dec/24 01:40 am>[1507748.069655]  [<ffffffff80249439>] ? worker_thread+0x0/0xe3
<Dec/24 01:40 am>[1507748.069655]  [<ffffffff8024becc>] ? kthread+0x47/0x73
<Dec/24 01:40 am>[1507748.069655]  [<ffffffff80234852>] ? schedule_tail+0x27/0x5f
<Dec/24 01:40 am>[1507748.069655]  [<ffffffff8020cd19>] ? child_rip+0xa/0x11
<Dec/24 01:40 am>[1507748.069655]  [<ffffffff8024be85>] ? kthread+0x0/0x73
<Dec/24 01:40 am>[1507748.069655]  [<ffffffff8020cd0f>] ? child_rip+0x0/0x11
<Dec/24 01:40 am>[1507748.069655] Code: 48 8d 7c 24 10 48 8d 73 18 fc 4d 8b 6c 24 18 b9 06 00 00 00 f3 a5 4c 89 a5 80 00 00 00 48 89 ef 48 8b 43 08 48 8b 13 48 89 42 08 <48> 89 10 48 89 5b 08 48 89 1b e8 d1 ed 36 00 48 8b 43 f8 48 83 
<Dec/24 01:40 am>[1507748.069655] RIP  [<ffffffff802488fd>] run_workqueue+0x77/0x1ed
<Dec/24 01:40 am>[1507748.069655]  RSP <ffff88063b293e80>
<Dec/24 01:40 am>[1507748.069655] ---[ end trace d2b44a22042f9805 ]---

<Dec/24 01:40 am>[1507761.949892] nfs: server ippc18 not responding, still trying
<Dec/24 01:41 am>[1507794.116723] ata1.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x6 frozen
<Dec/24 01:41 am>[1507794.124527] ata1.00: cmd ea/00:00:00:00:00/00:00:00:00:00/a0 tag 0
<Dec/24 01:41 am>[1507794.124528]          res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
<Dec/24 01:41 am>[1507794.139389] ata1.00: status: { DRDY }
<Dec/24 01:41 am>[1507794.143535] ata1: hard resetting link
<Dec/24 01:41 am>[1507794.480033] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
<Dec/24 01:41 am>[1507799.493344] ata1.00: qc timeout (cmd 0xec)
<Dec/24 01:41 am>[1507799.497890] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
<Dec/24 01:41 am>[1507799.504459] ata1.00: revalidation failed (errno=-5)
<Dec/24 01:41 am>[1507799.509803] ata1: hard resetting link
<Dec/24 01:41 am>[1507799.876696] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
<Dec/24 01:41 am>[1507809.902766] ata2.00: exception Emask 0x0 SAct 0x1 SErr 0x0 action 0x6 frozen
<Dec/24 01:41 am>[1507809.903379] ata1.00: qc timeout (cmd 0xec)
<Dec/24 01:41 am>[1507809.903386] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)
<Dec/24 01:41 am>[1507809.903388] ata1.00: revalidation failed (errno=-5)
<Dec/24 01:41 am>[1507809.903392] ata1: hard resetting link
<Dec/24 01:41 am>[1507809.931076] ata2.00: cmd 60/40:00:b1:05:03/00:00:00:00:00/40 tag 0 ncq 32768 in
<Dec/24 01:41 am>[1507809.931078]          res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
<Dec/24 01:41 am>[1507809.947109] ata2.00: status: { DRDY }
<Dec/24 01:41 am>[1507809.951224] ata2: hard resetting link
<Dec/24 01:41 am>[1507812.902775] ata2: SATA link up 3.0 Gbps (SStatus 123 SCont<Dec/24 01:41 am>

<Dec/24 01:41 am>[1507812.903400] ata1: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
<Dec/24 01:41 am>[1507817.902768] ata2.00: qc timeout (cmd 0xec)
<Dec/24 01:41 am>[1507817.907368] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x4)
<Dec/24 01:41 am>[1507817.914048] ata2.00: revalidation failed (errno=-5)
<Dec/24 01:41 am>[1507817.919434] ata2: hard resetting link
<Dec/24 01:41 am>[1507820.902776] ata2: SATA link up 3.0 Gbps (SStatus 123 SCont<Dec/24 01:41 am>
                 <Dec/24 01:41 am>[1507830.902766] ata2.00: qc timeout (cmd 0xec)
<Dec/24 01:41 am>[1507830.907323] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x4)
<Dec/24 01:41 am>[1507830.913908] ata2.00: revalidation failed (errno=-5)
<Dec/24 01:41 am>[1507830.919250] ata2: hard resetting link
<Dec/24 01:41 am>[1507830.903013] BUG: spinlock lockup on CPU#0, rpciod/0/1671, <Dec/24 01:41 am>
                 <Dec/24 01:41 am>[1507830.903013] Pid: 1671, comm: rpciod/0 Tainted: G      D    2.6.28-rc7-00105-gfeaf384 #4
<Dec/24 01:41 am>[1507830.903013] Call Trace:
<Dec/24 01:41 am>[1507830.903013]  <IRQ>  [<ffffffff803a103f>] _raw_spin_lock+0xfb/0x122
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff80589645>] xprt_init_autodisconnect+0x0/0x98
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff805b7758>] _spin_lock_irqsave+0x37/0x3f
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff802492ab>] __queue_work+0x10/0x32
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff802492ab>] __queue_work+0x10/0x32
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff80589645>] xprt_init_autodisconnect+0x0/0x98
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8024932c>] queue_work_on+0x37/0x40
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff80241e07>] run_timer_softirq+0x169/0x1dd
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8023e144>] __do_softirq+0x7a/0x13d

<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8020d07c>] call_softirq+0x1c/0x28
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8020e028>] do_softirq+0x2c/0x68
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8023e084>] irq_exit+0x3f/0x85
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8021e66d>] smp_apic_timer_interrupt+0x93/0xac
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8020cacb>] apic_timer_interrupt+0x6b/0x70
<Dec/24 01:41 am>[1507830.903013]  <EOI>  [<ffffffff805b76fe>] _spin_unlock_irq+0x21/0x22
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff803f3936>] tty_audit_exit+0x60/0x92
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8023c516>] do_exit+0x1b0/0x74d
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff805b83fa>] oops_begin+0x0/0x8d
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff805b795d>] error_exit+0x0/0x9a
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff802488fd>] run_workqueue+0x77/0x1ed
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff80249511>] worker_thread+0xd8/0xe3
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8024bfda>] autoremove_wake_function+0x0/0x2e
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff80249439>] worker_thread+0x0/0xe3
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8024becc>] kthread+0x47/0x73
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff80234852>] schedule_tail+0x27/0x5f
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8020cd19>] child_rip+0xa/0x11
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8024be85>] kthread+0x0/0x73
<Dec/24 01:41 am>[1507830.903013]  [<ffffffff8020cd0f>] child_rip+0x0/0x11
<Dec/24 01:41 am>[1507833.902778] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
<Dec/24 01:41 am>[1507842.903354] ata1.00: qc timeout (cmd 0xec)
<Dec/24 01:41 am>[1507842.907950] ata1.00: failed to IDENTIFY (I/O error, err_mask=0x4)

<Dec/24 01:41 am>[1507842.914597] ata1.00: revalidation failed (errno=-5)
<Dec/24 01:41 am>[1507842.920029] ata1.00: disabled
<Dec/24 01:41 am>[1507842.923489] ata1: hard resetting link
<Dec/24 01:41 am>[1507845.903359] ata1: SATA link up 3.0 Gbps (SStatus 123 SCont<Dec/24 01:41 am>
                 <Dec/24 01:41 am>[1507845.910057] ata1: EH complete
<Dec/24 01:41 am>[1507845.913479] sd 0:0:0:0: [sda] Result: hostbyte=DID_BAD_TAR<Dec/24 01:41 am>IVER_OK,SUGGEST_OK
                 <Dec/24 01:41 am>[1507845.922808] end_request: I/O error, dev sda, sector 96195185
<Dec/24 01:41 am>[1507845.928983] end_request: I/O error, dev sda, sector 96195185
<Dec/24 01:41 am>[1507845.932307] md: super_written gets error=-5, uptodate=0
<Dec/24 01:41 am>[1507845.932307] raid1: Disk failure on sda3, disabling device.
<Dec/24 01:41 am>[1507845.932307] raid1: Operation continuing on 1 devices.
<Dec/24 01:42 am>[1507863.902775] ata2.00: qc timeout (cmd 0xec)
<Dec/24 01:42 am>[1507863.907324] ata2.00: failed to IDENTIFY (I/O error, err_mask=0x4)
<Dec/24 01:42 am>[1507863.913897] ata2.00: revalidation failed (errno=-5)
<Dec/24 01:42 am>[1507863.919302] ata2.00: disabled
<Dec/24 01:42 am>[1507863.922742] ata2: hard resetting link
<Dec/24 01:42 am>[1507866.902771] ata2: SATA link up 3.0 Gbps (SStatus 123 SControl 300)
<Dec/24 01:42 am>[1507866.909457] ata2: EH complete
<Dec/24 01:42 am>[1507866.912951] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507866.922239] end_request: I/O error, dev sdb, sector 198065
<Dec/24 01:42 am>[1507866.928211] Read-error on swap-device (8:16:198073)
<Dec/24 01:42 am>[1507866.933557] Read-error on swap-device (8:16:198081)

<Dec/24 01:42 am>[1507866.938886] Read-error on swap-device (8:16:198089)
<Dec/24 01:42 am>[1507866.944238] Read-error on swap-device (8:16:198097)
<Dec/24 01:42 am>[1507866.949586] Read-error on swap-device (8:16:198105)
<Dec/24 01:42 am>[1507866.954995] Read-error on swap-device (8:16:198113)
<Dec/24 01:42 am>[1507866.960403] Read-error on swap-device (8:16:198121)
<Dec/24 01:42 am>[1507866.965802] Read-error on swap-device (8:16:198129)
<Dec/24 01:42 am>[1507866.971227] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507866.980626] end_request: I/O error, dev sdb, sector 96195185
<Dec/24 01:42 am>[1507866.986850] end_request: I/O error, dev sdb, sector 96195185
<Dec/24 01:42 am>[1507866.990174] md: super_written gets error=-5, uptodate=0
<Dec/24 01:42 am>[1507866.998796] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.008133] end_request: I/O error, dev sdb, sector 96195185
<Dec/24 01:42 am>[1507867.014352] end_request: I/O error, dev sdb, sector 96195185
<Dec/24 01:42 am>[1507867.017673] md: super_written gets error=-5, uptodate=0
<Dec/24 01:42 am>[1507867.026397] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.035706] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.041946] RAID1 conf printout:
<Dec/24 01:42 am>[1507867.042052] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042057] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042085] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042089] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042106] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK

<Dec/24 01:42 am>[1507867.042109] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042127] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042131] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042147] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042150] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042168] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042171] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042187] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042191] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042208] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042212] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042228] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042231] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042249] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042252] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042268] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042271] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042289] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042292] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042308] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042311] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042329] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK

<Dec/24 01:42 am>[1507867.042332] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042348] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042351] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042369] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042372] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042388] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042391] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042409] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042412] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042428] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042432] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042449] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042452] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042469] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042473] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042491] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042494] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042510] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042514] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042531] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042535] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042551] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK

<Dec/24 01:42 am>[1507867.042554] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042572] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042575] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042591] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042595] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042613] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042625] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042643] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042646] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042664] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042667] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042684] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042687] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042706] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042709] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042726] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042729] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042747] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042750] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042766] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042769] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042787] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK

<Dec/24 01:42 am>[1507867.042790] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042807] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042810] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042828] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042831] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042847] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042850] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042868] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042871] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042888] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042891] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042909] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042912] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042928] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042931] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042949] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042952] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042969] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042972] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.042990] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.042993] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.043010] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK

<Dec/24 01:42 am>[1507867.043013] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.043038] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.043041] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.043058] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.043061] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.043079] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.043082] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.043098] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.043102] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.043119] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.043123] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.043139] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.043142] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.043160] sd 1:0:0:0: [sdb] Result: hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
<Dec/24 01:42 am>[1507867.043163] end_request: I/O error, dev sdb, sector 73136425
<Dec/24 01:42 am>[1507867.886971]  --- wd:1 rd:2
<Dec/24 01:42 am>[1507867.890137]  disk 0, wo:1, o:0, dev:sda3
<Dec/24 01:42 am>[1507867.894539]  disk 1, wo:0, o:1, dev:sdb3
<Dec/24 01:42 am>[1507869.603310] BUG: soft lockup - CPU#11 stuck for 61s! [events/11:50]
<Dec/24 01:42 am>[1507869.603316] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
<Dec/24 01:42 am>[1507869.603316] CPU 11:
<Dec/24 01:42 am>[1507869.603316] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan

<Dec/24 01:42 am>[1507869.603316] Pid: 50, comm: events/11 Tainted: G      D    2.6.28-rc7-00105-gfeaf384 #4
<Dec/24 01:42 am>[1507869.603316] RIP: 0010:[<ffffffff8025bd5d>]  [<ffffffff8025bd5d>] smp_call_function_mask+0x163/0x1d7
<Dec/24 01:42 am>[1507869.603316] RSP: 0018:ffff88063ceb7d30  EFLAGS: 00000202
<Dec/24 01:42 am>[1507869.603316] RAX: 0000000000000020 RBX: 000000000000000b RCX: 0000000000000020
<Dec/24 01:42 am>[1507869.603316] RDX: ffffffff8081a300 RSI: 0000000000000020 RDI: 0000000000000000
<Dec/24 01:42 am>[1507869.603316] RBP: 000000000000000b R08: 0000000000000020 R09: ffff88063ceb7cf0
<Dec/24 01:42 am>[1507869.603316] R10: 0000000000000000 R11: ffffffff802230fe R12: 0000000000000001
<Dec/24 01:42 am>[1507869.603316] R13: ffffffff80257376 R14: 0000000000000069 R15: ffff88063ceac388
<Dec/24 01:42 am>[1507869.603316] FS:  0000000000000000(0000) GS:ffff880c3cc52900(0000) knlGS:0000000000000000
<Dec/24 01:42 am>[1507869.603316] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
<Dec/24 01:42 am>[1507869.603316] CR2: 00007fff666100c0 CR3: 0000000000201000 CR4: 00000000000006e0
<Dec/24 01:42 am>[1507869.603316] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/24 01:42 am>[1507869.603316] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/24 01:42 am>[1507869.603316] Call Trace:
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff80257376>] ? __lock_acquire+0x788/0x7f3
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff8022ecd2>] ? update_curr+0x93/0xc9
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff80218079>] ? mcheck_check_cpu+0x0/0x2f
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff8023dcbb>] ? on_each_cpu+0x10/0x22
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff80217997>] ? mcheck_timer+0x0/0x76
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff802179ab>] ? mcheck_timer+0x14/0x76
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff80248973>] ? run_workqueue+0xed/0x1ed
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff8024891d>] ? run_workqueue+0x97/0x1ed

<Dec/24 01:42 am>[1507869.603316]  [<ffffffff80249511>] ? worker_thread+0xd8/0xe3
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff8024bfda>] ? autoremove_wake_function+0x0/0x2e
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff80249439>] ? worker_thread+0x0/0xe3
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff8024becc>] ? kthread+0x47/0x73
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff80234852>] ? schedule_tail+0x27/0x5f
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff8020cd19>] ? child_rip+0xa/0x11
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff8024be85>] ? kthread+0x0/0x73
<Dec/24 01:42 am>[1507869.603316]  [<ffffffff8020cd0f>] ? child_rip+0x0/0x11
<Dec/24 01:43 am>[1507935.103303] BUG: soft lockup - CPU#11 stuck for 61s! [events/11:50]
<Dec/24 01:43 am>[1507935.103310] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
<Dec/24 01:43 am>[1507935.103310] CPU 11:
<Dec/24 01:43 am>[1507935.103310] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
<Dec/24 01:43 am>[1507935.103310] Pid: 50, comm: events/11 Tainted: G      D    2.6.28-rc7-00105-gfeaf384 #4
<Dec/24 01:43 am>[1507935.103310] RIP: 0010:[<ffffffff8025bd5d>]  [<ffffffff8025bd5d>] smp_call_function_mask+0x163/0x1d7
<Dec/24 01:43 am>[1507935.103310] RSP: 0018:ffff88063ceb7d30  EFLAGS: 00000202
<Dec/24 01:43 am>[1507935.103310] RAX: 0000000000000020 RBX: 000000000000000b RCX: 0000000000000020
<Dec/24 01:43 am>[1507935.103310] RDX: ffffffff8081a300 RSI: 0000000000000020 RDI: 0000000000000000
<Dec/24 01:43 am>[1507935.103310] RBP: 000000000000000b R08: 0000000000000020 R09: ffff88063ceb7cf0
<Dec/24 01:43 am>[1507935.103310] R10: 0000000000000000 R11: ffffffff802230fe R12: 0000000000000001
<Dec/24 01:43 am>[1507935.103310] R13: ffffffff80257376 R14: 0000000000000069 R15: ffff88063ceac388
<Dec/24 01:43 am>[1507935.103310] FS:  0000000000000000(0000) GS:ffff880c3cc52900(0000) knlGS:0000000000000000
<Dec/24 01:43 am>[1507935.103310] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b

<Dec/24 01:43 am>[1507935.103310] CR2: 00007fff666100c0 CR3: 0000000000201000 CR4: 00000000000006e0
<Dec/24 01:43 am>[1507935.103310] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/24 01:43 am>[1507935.103310] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/24 01:43 am>[1507935.103310] Call Trace:
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff80257376>] ? __lock_acquire+0x788/0x7f3
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff8022ecd2>] ? update_curr+0x93/0xc9
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff80218079>] ? mcheck_check_cpu+0x0/0x2f
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff8023dcbb>] ? on_each_cpu+0x10/0x22
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff80217997>] ? mcheck_timer+0x0/0x76
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff802179ab>] ? mcheck_timer+0x14/0x76
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff80248973>] ? run_workqueue+0xed/0x1ed
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff8024891d>] ? run_workqueue+0x97/0x1ed
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff80249511>] ? worker_thread+0xd8/0xe3
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff8024bfda>] ? autoremove_wake_function+0x0/0x2e
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff80249439>] ? worker_thread+0x0/0xe3
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff8024becc>] ? kthread+0x47/0x73
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff80234852>] ? schedule_tail+0x27/0x5f
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff8020cd19>] ? child_rip+0xa/0x11
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff8024be85>] ? kthread+0x0/0x73
<Dec/24 01:43 am>[1507935.103310]  [<ffffffff8020cd0f>] ? child_rip+0x0/0x11
<Dec/24 01:44 am>[1508000.599962] BUG: soft lockup - CPU#11 stuck for 61s! [events/11:50]
<Dec/24 01:44 am>[1508000.599971] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan

<Dec/24 01:44 am>[1508000.599971] CPU 11:
<Dec/24 01:44 am>[1508000.599971] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
<Dec/24 01:44 am>[1508000.599971] Pid: 50, comm: events/11 Tainted: G      D    2.6.28-rc7-00105-gfeaf384 #4
<Dec/24 01:44 am>[1508000.599971] RIP: 0010:[<ffffffff8025bd59>]  [<ffffffff8025bd59>] smp_call_function_mask+0x15f/0x1d7
<Dec/24 01:44 am>[1508000.599971] RSP: 0018:ffff88063ceb7d30  EFLAGS: 00000202
<Dec/24 01:44 am>[1508000.599971] RAX: 0000000000000020 RBX: 000000000000000b RCX: 0000000000000020
<Dec/24 01:44 am>[1508000.599971] RDX: ffffffff8081a300 RSI: 0000000000000020 RDI: 0000000000000000
<Dec/24 01:44 am>[1508000.599971] RBP: 000000000000000b R08: 0000000000000020 R09: ffff88063ceb7cf0
<Dec/24 01:44 am>[1508000.599971] R10: 0000000000000000 R11: ffffffff802230fe R12: 0000000000000001
<Dec/24 01:44 am>[1508000.599971] R13: ffffffff80257376 R14: 0000000000000069 R15: ffff88063ceac388
<Dec/24 01:44 am>[1508000.599971] FS:  0000000000000000(0000) GS:ffff880c3cc52900(0000) knlGS:0000000000000000
<Dec/24 01:44 am>[1508000.599971] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
<Dec/24 01:44 am>[1508000.599971] CR2: 00007fff666100c0 CR3: 0000000000201000 CR4: 00000000000006e0
<Dec/24 01:44 am>[1508000.599971] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/24 01:44 am>[1508000.599971] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/24 01:44 am>[1508000.599971] Call Trace:
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff80257376>] ? __lock_acquire+0x788/0x7f3
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff8022ecd2>] ? update_curr+0x93/0xc9
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff80218079>] ? mcheck_check_cpu+0x0/0x2f
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff8023dcbb>] ? on_each_cpu+0x10/0x22
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff80217997>] ? mcheck_timer+0x0/0x76
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff802179ab>] ? mcheck_timer+0x14/0x76

<Dec/24 01:44 am>[1508000.599971]  [<ffffffff80248973>] ? run_workqueue+0xed/0x1<Dec/24 01:44 am>
                 <Dec/24 01:44 am>[1508000.599971]  [<ffffffff8024891d>] ? run_workqueue+0x97/0x1ed
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff80249511>] ? worker_thread+0xd8/0xe3
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff8024bfda>] ? autoremove_wake_function+0x0/0x2e
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff80249439>] ? worker_thread+0x0/0xe3
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff8024becc>] ? kthread+0x47/0x73
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff80234852>] ? schedule_tail+0x27/0x5f
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff8020cd19>] ? child_rip+0xa/0x11
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff8024be85>] ? kthread+0x0/0x73
<Dec/24 01:44 am>[1508000.599971]  [<ffffffff8020cd0f>] ? child_rip+0x0/0x11
<Dec/24 01:45 am>[1508066.099956] BUG: soft lockup - CPU#11 stuck for 61s! [events/11:50]
<Dec/24 01:45 am>[1508066.099964] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
<Dec/24 01:45 am>[1508066.099964] CPU 11:
<Dec/24 01:45 am>[1508066.099964] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
<Dec/24 01:45 am>[1508066.099964] Pid: 50, comm: events/11 Tainted: G      D    2.6.28-rc7-00105-gfeaf384 #4
<Dec/24 01:45 am>[1508066.099964] RIP: 0010:[<ffffffff8025bd5d>]  [<ffffffff8025bd5d>] smp_call_function_mask+0x163/0x1d7
<Dec/24 01:45 am>[1508066.099964] RSP: 0018:ffff88063ceb7d30  EFLAGS: 00000202
<Dec/24 01:45 am>[1508066.099964] RAX: 0000000000000020 RBX: 000000000000000b RCX: 0000000000000020
<Dec/24 01:45 am>[1508066.099964] RDX: ffffffff8081a300 RSI: 0000000000000020 RDI: 0000000000000000
<Dec/24 01:45 am>[1508066.099964] RBP: 000000000000000b R08: 0000000000000020 R09: ffff88063ceb7cf0
<Dec/24 01:45 am>[1508066.099964] R10: 0000000000000000 R11: ffffffff802230fe R12: 0000000000000001
<Dec/24 01:45 am>[1508066.099964] R13: ffffffff80257376 R14: 0000000000000069 R15: ffff88063ceac388

<Dec/24 01:45 am>[1508066.099964] FS:  0000000000000000(0000) GS:ffff880c3cc52900(0000) knlGS:0000000000000000
<Dec/24 01:45 am>[1508066.099964] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
<Dec/24 01:45 am>[1508066.099964] CR2: 00007fff666100c0 CR3: 0000000000201000 CR4: 00000000000006e0
<Dec/24 01:45 am>[1508066.099964] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/24 01:45 am>[1508066.099964] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/24 01:45 am>[1508066.099964] Call Trace:
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff80257376>] ? __lock_acquire+0x788/0x7f3
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff8022ecd2>] ? update_curr+0x93/0xc9
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff80218079>] ? mcheck_check_cpu+0x0/0x2f
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff8023dcbb>] ? on_each_cpu+0x10/0x22
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff80217997>] ? mcheck_timer+0x0/0x76
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff802179ab>] ? mcheck_timer+0x14/0x76
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff80248973>] ? run_workqueue+0xed/0x1ed
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff8024891d>] ? run_workqueue+0x97/0x1ed
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff80249511>] ? worker_thread+0xd8/0xe3
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff8024bfda>] ? autoremove_wake_function+0x0/0x2e
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff80249439>] ? worker_thread+0x0/0xe3
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff8024becc>] ? kthread+0x47/0x73
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff80234852>] ? schedule_tail+0x27/0x5f
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff8020cd19>] ? child_rip+0xa/0x11
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff8024be85>] ? kthread+0x0/0x73
<Dec/24 01:45 am>[1508066.099964]  [<ffffffff8020cd0f>] ? child_rip+0x0/0x11

<Dec/24 01:46 am>[1508131.596616] BUG: soft lockup - CPU#11 stuck for 61s! [events/11:50]
<Dec/24 01:46 am>[1508131.596624] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
<Dec/24 01:46 am>[1508131.596624] CPU 11:
<Dec/24 01:46 am>[1508131.596624] Modules linked in: w83793 hwmon_vid autofs4 e100 i2c_i801 i2c_core iTCO_wdt e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
<Dec/24 01:46 am>[1508131.596624] Pid: 50, comm: events/11 Tainted: G      D    2.6.28-rc7-00105-gfeaf384 #4
<Dec/24 01:46 am>[1508131.596624] RIP: 0010:[<ffffffff8025bd5d>]  [<ffffffff8025bd5d>] smp_call_function_mask+0x163/0x1d7
<Dec/24 01:46 am>[1508131.596624] RSP: 0018:ffff88063ceb7d30  EFLAGS: 00000202
<Dec/24 01:46 am>[1508131.596624] RAX: 0000000000000020 RBX: 000000000000000b RCX: 0000000000000020
<Dec/24 01:46 am>[1508131.596624] RDX: ffffffff8081a300 RSI: 0000000000000020 RDI: 0000000000000000
<Dec/24 01:46 am>[1508131.596624] RBP: 000000000000000b R08: 0000000000000020 R09: ffff88063ceb7cf0
<Dec/24 01:46 am>[1508131.596624] R10: 0000000000000000 R11: ffffffff802230fe R12: 0000000000000001
<Dec/24 01:46 am>[1508131.596624] R13: ffffffff80257376 R14: 0000000000000069 R15: ffff88063ceac388
<Dec/24 01:46 am>[1508131.596624] FS:  0000000000000000(0000) GS:ffff880c3cc52900(0000) knlGS:0000000000000000
<Dec/24 01:46 am>[1508131.596624] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
<Dec/24 01:46 am>[1508131.596624] CR2: 00007fff666100c0 CR3: 0000000000201000 CR4: 00000000000006e0
<Dec/24 01:46 am>[1508131.596624] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
<Dec/24 01:46 am>[1508131.596624] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
<Dec/24 01:46 am>[1508131.596624] Call Trace:
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff80257376>] ? __lock_acquire+0x788/0x7f3
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff8022ecd2>] ? update_curr+0x93/0xc9
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff80218079>] ? mcheck_check_cpu+0x0/0x2f
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff8023dcbb>] ? on_each_cpu+0x10/0x22

<Dec/24 01:46 am>[1508131.596624]  [<ffffffff80217997>] ? mcheck_timer+0x0/0x76
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff802179ab>] ? mcheck_timer+0x14/0x76
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff80248973>] ? run_workqueue+0xed/0x1ed
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff8024891d>] ? run_workqueue+0x97/0x1ed
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff80249511>] ? worker_thread+0xd8/0xe3
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff8024bfda>] ? autoremove_wake_functi<Dec/24 01:46 am>
                 <Dec/24 01:46 am>[1508131.596624]  [<ffffffff80249439>] ? worker_thread+0x0/0xe3
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff8024becc>] ? kthread+0x47/0x73
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff80234852>] ? schedule_tail+0x27/0x5f
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff8020cd19>] ? child_rip+0xa/0x11
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff8024be85>] ? kthread+0x0/0x73
<Dec/24 01:46 am>[1508131.596624]  [<ffffffff8020cd0f>] ? child_rip+0x0/0x11

File erased !

Last modified 12 years ago Last modified on Dec 24, 2014, 9:39:43 AM

Note: See TracWiki for help on using the wiki.

Download in other formats:

Plain Text