<Oct/07 07:54 am>[252314.882127] HARDWARE ERROR
<Oct/07 07:54 am>[252314.882127] CPU 7: Machine Check Exception:                5 Bank 0: b200004000000800
<Oct/07 07:54 am>[252314.884261] RIP !INEXACT! 33:<00007f84d3145975>
<Oct/07 07:54 am>[252314.884261] TSC 2dcb641a09418
<Oct/07 07:54 am>[252314.884261] This is not a software problem!
<Oct/07 07:54 am>[252314.884261] Run through mcelog --ascii to decode and contact your hardware vendor
<Oct/07 07:54 am>[252314.884261]



[cindy@uku ~]$ /usr/sbin/mcelog --p4 --ascii < err.txt 
HARDWARE ERROR. This is *NOT* a software problem!
Please contact your hardware vendor
CPU 7 BANK 0 TSC 2dcb641a09418 
MCG status:RIPV MCIP 
MCi status:
Uncorrected error
Error enabled
Processor context corrupt
MCA:BUS Level-0 Originated-request Generic Memory-access Request-timeout Error
Model:
STATUS b200004000000800 MCGSTATUS 5




<Oct/07 07:54 am>[252314.884261] HARDWARE ERROR
<Oct/07 07:54 am>[252314.884261] CPU 5: Machine Check Exception:                4 Bank 0: b200004000000800
<Oct/07 07:54 am>[252314.922716] TSC 2dcb641a09430
<Oct/07 07:54 am>[252314.922716] This is not a software problem!
<Oct/07 07:54 am>[252314.922716] Run through mcelog --ascii to decode and contact your hardware vendor
<Oct/07 07:54 am>[252314.922716]





[cindy@uku ~]$ /usr/sbin/mcelog --p4 --ascii < err.txt 
HARDWARE ERROR. This is *NOT* a software problem!
Please contact your hardware vendor
CPU 5 BANK 0 TSC 2dcb641a09430 
MCG status:MCIP 
MCi status:
Uncorrected error
Error enabled
Processor context corrupt
MCA:BUS Level-0 Originated-request Generic Memory-access Request-timeout Error
Model:
STATUS b200004000000800 MCGSTATUS 4




<Oct/07 07:54 am>[252314.922716] HARDWARE ERROR
<Oct/07 07:54 am>[252314.922716] CPU 5: Machine Check Exception:                4 Bank 5: b200120014040400
<Oct/07 07:54 am>[252314.922716] TSC 2dcb641a0a3d8
<Oct/07 07:54 am>[252314.922716] This is not a software problem!
<Oct/07 07:54 am>[252314.922716] Run through mcelog --ascii to decode and contact your hardware vendor
<Oct/07 07:54 am>[252314.922716]

[cindy@uku ~]$ /usr/sbin/mcelog --p4 --ascii < err.txt 
HARDWARE ERROR. This is *NOT* a software problem!
Please contact your hardware vendor
CPU 5 BANK 5 TSC 2dcb641a0a3d8 
MCG status:MCIP 
MCi status:
Uncorrected error
Error enabled
Processor context corrupt
MCA:Internal Timer error
STATUS b200120014040400 MCGSTATUS 4




<Oct/07 07:54 am>[252314.922716] HARDWARE ERROR
<Oct/07 07:54 am>[252314.922716] CPU 7: Machine Check Exception:                5 Bank 5: b200120020080400
<Oct/07 07:54 am>[252314.922716] RIP !INEXACT! 33:<00007f84d3145975>
<Oct/07 07:54 am>[252314.922716] TSC 2dcb641a0a388
<Oct/07 07:54 am>[252314.922716] This is not a software problem!
<Oct/07 07:54 am>[252314.922716] Run through mcelog --ascii to decode and contact your hardware vendor


[cindy@uku ~]$ /usr/sbin/mcelog --p4 --ascii < err.txt 
HARDWARE ERROR. This is *NOT* a software problem!
Please contact your hardware vendor
CPU 7 BANK 5 TSC 2dcb641a0a388 
MCG status:RIPV MCIP 
MCi status:
Uncorrected error
Error enabled
Processor context corrupt
MCA:Internal Timer error
STATUS b200120020080400 MCGSTATUS 5


<Oct/07 07:54 am>[252314.922716] Kernel panic - not syncing: Machine check
<Oct/07 07:54 am>[252314.922716] ------------[ cut here ]------------
<Oct/07 07:54 am>[252314.922716] WARNING: at kernel/smp.c:333 smp_call_function_mask+0x37/0x1d7()
<Oct/07 07:54 am>[252314.922716] Modules linked in: coretemp w83627hf w83793 hwmon_vid k8temp autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr
aic7xxx scsi_wait_scan

<Oct/07 07:54 am>[252314.922716] Pid: 25746, comm: ppSub Tainted: G   M    W  2.6.28-rc7-00105-gfeaf384 #4
<Oct/07 07:54 am>[252314.922716] Call Trace:
<Oct/07 07:54 am>[252314.922716]  <#MC>  [<ffffffff802397db>] warn_on_slowpath+0x51/0x6d
<Oct/07 07:54 am>[252314.922716]  [<ffffffff803eec86>] notify_update+0x2b/0x30
<Oct/07 07:54 am>[252314.922716]  [<ffffffff8025bc31>] smp_call_function_mask+0x37/0x1d7
<Oct/07 07:54 am>[252314.922716]  [<ffffffff80264013>] crash_kexec+0x17/0xef
<Oct/07 07:54 am>[252314.922716]  [<ffffffff802640e2>] crash_kexec+0xe6/0xef
<Oct/07 07:54 am>[252315.092717]  [<ffffffff8021d53e>] native_smp_send_stop+0x1a/0x26
<Oct/07 07:54 am>[252315.099374]  [<ffffffff80239896>] panic+0x95/0x13f
<Oct/07 07:54 am>[252315.104985]  [<ffffffff80239d5d>] release_console_sem+0x3e/0x1a5
<Oct/07 07:54 am>[252315.111648]  [<ffffffff80239d5d>] release_console_sem+0x3e/<Oct/07 07:54 am>
<Oct/07 07:54 am>[252315.114983]  [<ffffffff80239d5d>] release_console_sem+0x3e/0x1a5
<Oct/07 07:54 am>[252315.121648]  [<ffffffff805b9e66>] __atomic_notifier_call_chain+0x74/0x83
<Oct/07 07:54 am>[252315.131649]  [<ffffffff805b9df2>] __atomic_notifier_call_chain+0x0/0x83
<Oct/07 07:54 am>[252315.131649]  [<ffffffff8023969e>] oops_enter+0x9/0x10
<Oct/07 07:54 am>[252315.131649]  [<ffffffff80217b55>] mce_log+0x0/0x7f
<Oct/07 07:54 am>[252315.148315]  [<ffffffff80217eef>] do_machine_check+0x2d5/0x378
<Oct/07 07:54 am>[252315.148315]  [<ffffffff8020d04f>] machine_check+0x7f/0x90
<Oct/07 07:54 am>[252315.148315]  <<EOE>> <4>---[ end trace 4eaa2a86a8e2da22 ]--

## On reboot
ipp029 login: [   96.784791]
[   96.784791] HARDWARE ERROR
[   96.784791] CPU 5: Machine Check Exception:                4 Bank 0: b200004000000800
[   96.784791] TSC 87ce2248f0
[   96.784791] This is not a software problem!
[   96.784791] Run through mcelog --ascii to decode and contact your hardware vendor
[   96.784791]
[   96.784791] HARDWARE ERROR
[   96.784791] CPU 7: Machine Check Exception:                5 Bank 0: b200004000000800
[   96.784791] RIP !INEXACT! 10:<ffffffff80212604> {mwait_idle+0x41/0x44}
[   96.784791] TSC 87ce2248b8
[   96.784791] This is not a software problem!
[   96.784791] Run through mcelog --ascii to decode and contact your hardware vendor
[   96.784791]
[   96.784791] HARDWARE ERROR
[   96.784791] CPU 7: Machine Check Exception:                5 Bank 3: 942000160004010a
[   96.784791] RIP !INEXACT! 10:<ffffffff80212604> {mwait_idle+0x41/0x44}
[   96.784791] TSC 87ce225740 ADDR 3f7b32dc0
[   96.784791] This is not a software problem!
[   96.784791] Run through mcelog --ascii to decode and contact your hardware vendor
[   96.784791]
[   96.784791] HARDWARE ERROR
[   96.784791] CPU 5: Machine Check Exception:                4 Bank 3: 942000160004010a
[   96.784791] TSC 87ce2256f0 ADDR 3f7b32dc0
[   96.784791] This is not a software problem!
[   96.784791] Run through mcelog --ascii to decode and contact your hardware vendor
[   96.784791]
[   96.784791] HARDWARE ERROR
[   96.784791] CPU 7: Machine Check Exception:                5 Bank 5: b200120020080400
[   96.784791] RIP !INEXACT! 10:<ffffffff80212604> {mwait_idle+0x41/0x44}
[   96.784791] TSC 87ce225a60
[   96.784791] This is not a software problem!
[   96.784791] Run through mcelog --ascii to decode and contact your hardware vendor
[   96.784791]
[   96.784791] HARDWARE ERROR
[   96.784791] CPU 5: Machine Check Exception:                4 Bank 5: b200120014040400
[   96.784791] TSC 87ce225f48
[   96.784791] This is not a software problem!
[   96.784791] Run through mcelog --ascii to decode and contact your hardware vendor
[   96.784791] Kernel panic - not syncing: Machine check
[   96.784791] ------------[ cut here ]------------
[   96.784791] WARNING: at kernel/smp.c:333 smp_call_function_mask+0x37/0x1d7()
[   96.784791] Modules linked in: coretemp w83627hf w83793 hwmon_vid k8temp autofs4 i2c_i801 e1000e iTCO_wdt i2c_core tg3 libphy e1000 xfs dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
[   96.784791] Pid: 9032, comm: perl Tainted: G   M    W  2.6.28-rc7-00105-gfeaf384 #4
[   96.784791] Call Trace:
[   96.784791]  <#MC>  [<ffffffff802397db>] warn_on_slowpath+0x51/0x6d
[   96.784791]  [<ffffffff803eec86>] notify_update+0x2b/0x30
[   96.784791]  [<ffffffff8025bc31>] smp_call_function_mask+0x37/0x1d7
[   96.784791]  [<ffffffff80264013>] crash_kexec+0x17/0xef
[   96.784791]  [<ffffffff802640e2>] crash_kexec+0xe6/0xef
[   96.784791]  [<ffffffff80239d5d>] release_console_sem+0x3e/0x1a5
[   96.784791]  [<ffffffff8025fbf4>] sprint_symbol+0x9f/0xaa
[   96.784791]  [<ffffffff8021d53e>] native_smp_send_stop+0x1a/0x26
[   96.784791]  [<ffffffff80239896>] panic+0x95/0x13f
[   96.784791]  [<ffffffff80239d5d>] release_console_sem+0x3e/0x1a5
[   96.784791]  [<ffffffff80239d5d>] release_console_sem+0x3e/0x1a5
[   96.784791]  [<ffffffff80239d5d>] release_console_sem+0x3e/0x1a5
[   96.784791]  [<ffffffff805b9e66>] __atomic_notifier_call_chain+0x74/0x83
[   96.784791]  [<ffffffff805b9df2>] __atomic_notifier_call_chain+0x0/0x83
[   96.784791]  [<ffffffff80217b55>] mce_log+0x0/0x7f
[   96.784791]  [<ffffffff80217eef>] do_machine_check+0x2d5/0x378
[   96.784791]  [<ffffffff8020d04f>] machine_check+0x7f/0x90
[   96.784791]  <<EOE>> <4>---[ end trace 4eaa2a86a8e2da22 ]---