(Up to PS1 IPP Czar Logs)

Monday : 2012-11-03

Serge is czar

• 06:30 (Serge): No observation last night.
• 10:00 (Serge): Stopping replication on ippdb02 and starting backup to ippc62 (/export/ippc62.1/backup_nebulous/20121203)
• 10:30 (Serge): Power cycled ipp012. The not-so-interesting apparent fault is the following:

  <Dec/03 01:40 am>ipp012 login: [2201848.347410] BUG: unable to handle kernel paging request at ffffffff800073d4
  <Dec/03 01:40 am>[2201848.349821] IP: [<ffffffff80589612>] xprt_autoclose+0x19/0x4c
  <Dec/03 01:40 am>[2201848.349821] PGD 203067 PUD 207063 PMD 0 
  <Dec/03 01:40 am>[2201848.349821] Oops: 0000 [#1] SMP 
  <Dec/03 01:40 am>[2201848.349821] last sysfs file: /sys/class/i2c-adapter/i2c-0/0-002f/temp6_alarm
  <Dec/03 01:40 am>[2201848.349821] CPU 0 
  <Dec/03 01:40 am>[2201848.349821] Modules linked in: w83793 hwmon_vid autofs4 i2c_i801 iTCO_wdt i2c_core e1000e tg3 libphy e1000 xfs 
  dm_snapshot dm_mirror dm_region_hash dm_log aacraid 3w_9xxx 3w_xxxx atp870u arcmsr aic7xxx scsi_wait_scan
  <Dec/03 01:40 am>[2201848.349821] Pid: 1522, comm: rpciod/0 Tainted: G        W  2.6.28-rc7-00105-gfeaf384 #4
  <Dec/03 01:40 am>[2201848.349821] RIP: 0010:[<ffffffff80589612>]  [<ffffffff80589612>] xprt_autoclose+0x19/0x4c
  <Dec/03 01:40 am>[2201848.349821] RSP: 0018:ffff88063c8ede60  EFLAGS: 00010246
  <Dec/03 01:40 am>[2201848.349821] RAX: ffffffff80007374 RBX: ffff8800697d0578 RCX: 0000000000000000
  <Dec/03 01:40 am>[2201848.349821] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff8800697d0000
  <Dec/03 01:40 am>[2201848.349821] RBP: ffff88033cc09a00 R08: 0000000000000001 R09: 0000000000000000
  <Dec/03 01:40 am>[2201848.349821] R10: 0000000000000000 R11: 0000000000000001 R12: ffff8800697d0578
  <Dec/03 01:40 am>[2201848.349821] R13: ffff8800697d0000 R14: ffffffff808b85a0 R15: 0000000000000000
  
  <Dec/03 01:40 am>[2201848.349821] FS:  0000000000000000(0000) GS:ffffffff80872d80(0000) knlGS:0000000000000000
  <Dec/03 01:40 am>[2201848.349821] CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
  <Dec/03 01:40 am>[2201848.349821] CR2: ffffffff800073d4 CR3: 0000000043d72000 CR4: 00000000000006e0
  <Dec/03 01:40 am>[2201848.349821] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  <Dec/03 01:40 am>[2201848.349821] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  <Dec/03 01:40 am>[2201848.349821] Process rpciod/0 (pid: 1522, threadinfo ffff88063c8ec000, task ffff88063ccc5b20)
  <Dec/03 01:40 am>[2201848.349821] Stack:
  <Dec/03 01:40 am>[2201848.349821]  ffff88063c8ede90 ffff8800697d0578 ffffffff805895f9 ffffffff80248973
  <Dec/03 01:40 am>[2201848.349821]  ffffffff8024891d 0000000000000046 0000000000000000 0000000000000000
  <Dec/03 01:40 am>[2201848.349821]  0000000000000000 ffff88033cc09a00 ffff88063e43bbf0 ffff88033cc09a00
  <Dec/03 01:40 am>[2201848.349821] Call Trace:
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff805895f9>] ? xprt_autoclose+0x0/0x4c
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff80248973>] ? run_workqueue+0xed/0x1ed
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff8024891d>] ? run_workqueue+0x97/0x1ed
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff80249511>] ? worker_thread+0xd8/0xe3
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff8024bfda>] ? autoremove_wake_function+0x0/0x2e
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff80249439>] ? worker_thread+0x0/0xe3
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff8024becc>] ? kthread+0x47/0x73
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff80234852>] ? schedule_tail+0x27/0x5f
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff8020cd19>] ? child_rip+0xa/0x11
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff8024be85>] ? kthread+0x0/0x73
  <Dec/03 01:40 am>[2201848.349821]  [<ffffffff8020cd0f>] ? child_rip+0x0/0x11
  
  <Dec/03 01:40 am>[2201848.349821] Code: 89 f7 e8 59 dd 02 00 89 d8 5b 5d 41 5c 41 5d 41 5e c3 41 55 4c 8d af 88 fa ff ff 41 54 53 48 
  8b 87 90 fa ff ff 48 89 fb 4c 89 ef <ff> 50 60 f0 80 63 d0 f7 4c 8d a3 98 00 00 00 4c 89 e7 e8 43 de 
  <Dec/03 01:40 am>[2201848.349821] RIP  [<ffffffff80589612>] xprt_autoclose+0x19/0x4c
  <Dec/03 01:40 am>[2201848.349821]  RSP <ffff88063c8ede60>
  <Dec/03 01:40 am>[2201848.349821] CR2: ffffffff800073d4
  <Dec/03 01:40 am>[2201848.349821] ---[ end trace 5dc435962a52df42 ]---
  

Tuesday : YYYY.MM.DD

Wednesday : YYYY.MM.DD

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD