* 2007-03-23T05:51:14 IDE Channel 24 Device Failed  
  * 2007-03-23T05:51:14 IDE Channel 22 Device Removed  
  * 2007-03-23T05:51:14 IDE Channel 23 Device Removed  
  * 2007-03-23T05:54:44 IDE Channel 1 Device Removed  
  * 2008-07-10T10:15:00 - forcedeth NIC failure - RPC errors on console
  * 2008-07-15T07:00:00 approx forecdeth problem power cycled by bills
  * 2008-08-11T17:23:32 forecdeth problem power cycled by gavin
  * 2009-11-17T05:14:47 IDE Channel 1 Reading Error (replaced 2009-11-17T14:12:40)
  * 2009-12-21905:51:08 IDE Channel 17 Device Failed (replaced 2009-12-22T16:33:35)
  * 2010-04-28T08:06:57	IDE Channel 6 Device Failed (replaced 2010-04-28T10:35:21)
  * 2010-05-31T22:52:03	IDE Channel 20 Device Failed (replaced 2010-06-01T13:26:54)
  * 2010-06-27T17:04:06 IDE Channel 12 Device Failed (replaced 2010-06-28T09:31:38)
  * 2010-07-17 experiencing 2 failed disk (2,7; rebuilding disk 2 first) [[br]]
    2010-7-T18T06:30:48 IDE Channel 2 Device Failed (replaced 2010-07-19T08:40:14) [[br]]
    2010-07-17T01:51:00 IDE Channel 7 Device Failed (replaced 2010-07-19T15:28:03)
  * 2010-07-25T17:56:4  IDE Channel 11 Device Failed (replaced 2010-07-26T08:50:43)
  * 2010-7-27T01:11:25  IDE Channel 16 Device Failed (replaced 20107-07-27T10:02:37)
  * 2010-08-13T01:43:39 IDE Channel 5 Device Failed (replaced 2010-08-13T10:20:28)
  * 2010-08-17T00:32:58 IDE Channel 14 Device Failed (replaced 2010-08-17T09:41:03)
  * 2010-08-25T08:48:06 IDE Channel 9 Device Failed (replaced 2010-08-25T10:19:47)
  * 2010-9-7  23:46:55 Controller#1(PCI) IDE Channel 18 Device Failed
  * 2010-10-23T01:51:56 IDE Channel 14 Device Failed (replaced 2010-10-25T)
  * 2010-11-7  3:10:18  Channel # 4 Device Failed replaced rebuilt
  * 2010-12-01T20:40    System became unresponsive. Nothing on console. Power cycled by bills 2010-12-02T642
  * 2010-12-14T17:18:50 IDE Channel 15 Device Failed (replaced 2010-12-14T17:50:49)
  * 2011-2-14  5:30:34 Controller # 1(PCI) IDE Channel # 19 Device Failed
  * 2011-2-24  0:31:30 Controller# 1(PCI) IDE Channel # 9 Device Failed
  * 2011-4-23  10:5:29 Controller # 1(PCI) IDE Channel # 9 Device Failed
  * 2011-4-22  6:15:34 Controller#1(PCI) IDE Channel # 5 Device Failed
  * 2012-09-06 14:00 rebooted due to mount problem (see czarlog [wiki:PS1_IPP_Czarlog_20120903]
  * 2012-09-21 12:00 rebooted and changed swapiness to see if helps swap use (see [wiki:PS1_IPP_Czarlog_20120917]
  * 2012-09-27 05:00 rebooted due to mount problem (see czarlog [wiki:PS1_IPP_Czarlog_20120924]
  * 2012-09-28 11:45 rebooted due to mount problem (see czarlog [wiki:PS1_IPP_Czarlog_20120924]
  * 2012-10-12 18:20 power cycled after became unresponsive [wiki:Ipp011-crash-20121012]
  * 2012-11-24 21:39 ditto [wiki:ipp011-crash-20121124]
  * 2012-11-29 02:36 [wiki:ipp011-crash-20121129] -- will need to power cycle after nightly science done, cannot log in but disks are accessible 
  * 2012-12-08 17:45 [wiki:ipp011-crash-20121208] -- like before, disks accessible still cannot login.  power cycled