* 2008-09-22T04:00 (approx) [wiki:Ipp033-crash-2008-09-22 kernel oops] * 2009-10-02T13:48:32 IDE Channel 10 Reading Error (reseated) * 2009-11-12T22:27:07 IDE Channel 10 Reading Error (replaced 2009-11-13T09:55:36) * 2010-01-05T16:26:21 system unresponsive, ganglia [attachment:ipp033-loadgraph-2010-01-05.png load], [attachment:ipp033-memorygraph-2010-01-05.png memory], [attachment:ipp033-cpugraph-2010-01-05.png cpu], power cycled by gavin * 2010-08-04T15:25:03 IDE Channel 2 Reading Error (replaced 2010-08-04T16:31:53) * 2011-06-28T11:50:04 network unresponsive, nothing on console, power cycled by gavin * 2011-7-5 17:17:21 Controller # 1(PCI) IDE Channel # 4 Reading Error Drive replaced * 2011-11-10 6:26:56 Controller 1(PCI) IDE Channel 9 Reading Error NOT REPLACED * 2011-9-17 12:32:49 Controller#1(PCI) IDE Channel # 7 Device Failed REPLACED * 2012-2-17 14:21:24 Controller#1(PCI) IDE Channel #16 Device Failed REPLACED * Oct 6 05:50:05 ipp033 archttp64: [WARNING]: ArecaRaidCtrl Event(URGENT): 2012-10-6 4:40:43 Controller#1(PCI) IDE Channel #24 Device Failed (replaced: Oct 6 11:28:09) * Note: a 1TB WD drive failed. We replaced it with a 1TB Seagate drive, but it didn't recognize that drive, so I replaced with a new 2TB drive. * Note: the time reported by the Areca RAID controller is off by more than an hour! * 2012-11-05 ~19:00 system crash, no info on display or in /var/log/messages [wiki:ipp033-crash-20121105]. NFS did not come up right, had to restart. ganglia didn't recognize as back up for ~10 minutes..