IPP Progress Report for the week 2011.10.17 - 2011.10.21

(Up to IPP Progress Reports)

Eugene Magnier

We have been dealing with a number of hardware problems this week, now with machines that have not had problems before. We had several of the ippcXX nodes crash (these have in the past been quite reliable). ippc22 now seems to be unusable for processing -- it crashes if any load is put on it. In addition, ipp029 has been having trouble since the previous week -- we (Cindy) tried replacing memory modules with no luck. It also cannot be used for any processing or as a nebulous target -- I now suspect a flaky raid card. Up at the ATRC, one of the raids on ippb00 was behaving badly; we have a replacement raid card on order. Finally, ipp036 crashed hard on Thursday and could not be rebooted. With the training going on on Oahu and Cindy needed at the summit on Friday, we could not get to this until Monday. Cindy replaced the motherboard -- it seems a cpu fan died and took out the mobo as well. There is some suspicion that we have having trouble with heat management at MHPCC, but we do not have enough data monitoring to know. We will update ganglia with per-machine temperatures.

On my side, between blue Ls, illness and czaring I did not get much development done this week, except to release a new (2MASS-based calibration) DVO for 3pi.

Serge Chastel

  • ippdb02 replication
  • PSPS training

Heather Flewelling

  • on vacation

Roy Henderson

  • Bulk of week was PSPS training with Sue
  • PSVO
    • some GUI fixes, e.g. graying-out query pane while query is running, new icons etc
    • fixed bug where temp VOTable files were not being deleted
  • czartool: hacked slightly to include 'update' as well as 'new' to give reliable stats for LAP processing
  • sick on Friday

Mark Huber

  • Czar Monday, Tuesday, Friday + LAP
  • MD04 deep test stack g-band setup and running, MD07 refstack prep
  • ppStack testing: tracked down ~300 input to limit in executing shell, tracing excess memory usage and generation of overly large PSF in convolved stacks
  • ppSub testing continued
  • raid throughput and psphotstack testing while reprocessing stalled

Bill Sweeney

  • Debugged several strange problems with updates triggered by postage stamp requests. There are still some wormholes that dependency checking gets lost in. (The cluster problems did not help)
  • Tried to install the IPP on a new system at MPIA. The version of the O/S that they have is causing problems. Need help from Gene.
  • during some time when work was stopped because nebulous was turned off pressed the Ubuntu upgrade button for my desktop. This turned out to be a very expensive choice. It seems that the developers are attempting to make Ubuntu look like some Microsoft Windows or perhaps like MacOS. Anyways what they have done is unusable. It took me the better part of the day getting things set up again. Sigh
  • Figured out why ppStack doesn't work off of the IPP trunk. Gene integrated a fix.
  • vacation 1 day (moving residence).

Chris Waters