<pre>

Status as of 2008-11-14

Nodes down:

None.

Notes:

• ipp015 is rebuilding disk #16

Known issues:

• random system crashes under heavy load of nodes, occasionally with a printk() of "do_IRQ: X.XXX" which appears to be caused by a hardware interrupt that does not have a handler (device driver) for it.
• ipp004 - disk bay #12 dead (system is usable)
• ipp005 - seems to have random instability issues beyond the do_IRQ issue of the other nodes
• ipp008 - disk bay #7 dead (system is usable)
• ipp013 & ipp016 have dead fans but it is not the fans themselves (tried replacing them). It appears to be the cabling that leads up to the fan modules that's gone bad.
• forcedeth.c max_interrupt_work is still too conservative (requires a cluster restart to change).
• ipp025 - suspect bad memory module (system is usable)
• Our contact at MHPCC, Brad Thomas, notified me that an alarm from cabinet 4 managed power strip had been set off last evening due to high loads on nodes (ipp017, 030-036). Please consider distributing your jobs evenly across ipp production nodes.

Below is the layout at MHPCC. cabinet0: ipp020-21 cabinet1: ipp012-014,008,016,018-019 cabinet2: ipp004-007,015,009-011 cabinet3: ipp023-029 cabinet4: ipp030-036,017

​http://kiawe.ifa.hawaii.edu/IPPwiki/index.php/Production_Cluster_Status

</pre>