PS1 IPP Czar Logs for the week 2015.02.23 - 2015.03.01

(Up to PS1 IPP Czar Logs)

Monday : 2015.02.23

07:30 EAM : I should have restarted stdscience last night -- it was running slowly in the night and is now somewhat behind. I'm restarting it now.

• 10:30 MEH: restarting pstamp, >300k jobs
• 10:50 MEH: ipps (host m0,1) and ippx037-044 (hosts x0/1b) off in stdlocal and staticsky, leave ippx (x0,1,2,3) as allocated in stdlocal and staticsky
  • leave these in staticsky until ~tuesday

Tuesday : 2014.02.24

• 13:30 EAM: I've stopped ippsky/staticsky so I can rebuild psModules and psphot for ipp-20150115. I have updated the code to provide psf.e1,e2 values for the lensing parameters, and I'd like to re-run the full-force analysis with this change. I'm going to run full-force only under ippsky while working on the ppSub errors mentioned by Chris. I'll put additional x-node servers into stdlocal for now.
• 13:55 EAM: ipp056 nfs is wedged. it seems to have gotten stuck on ipp078, but nothing seems to clear it. i'm attempting to reboot, and may need to power cycle.
• 14:05 EAM: regular ipp services restarted in the normal fashion. stdlocal restarted, with 7* m0, m1, x0b, x1b for now. ippsky still running sas fforce. when this is done I'll turn on staticsky with just x0 and x1 nodes.
• 16:00 EAM: stdlocal seems to want lots of jobs using stsci14. the 35 jobs limit is thus throttling the work there. rather than tweak this, I've taken the x3 nodes from stdlocal and put them in the full-force analysis for sas. when that is done, we can use them depending on the status of warps on stdlocal.

Wednesday : YYYY.MM.DD

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD