PS1_IPP_Czarlog_20140721 – Pan-STARRS IPP

Context Navigation

Version 13 (modified by Mark Huber, 12 years ago) ( diff )
--

11:20 EAM : I re-labeled the LAP.ThreePi.20130717 staticsky runs in dense fields back to the standard name (from variants with .dense, .100k, etc appended). I queued the unqueued skycal skycells from these regions. I've also stopped and restarted the skycal pantasks (running as ipptest@ippc04) since pcontrol was getting sluggish.

18:00 MEH: MOPS needs another sample reprocessing diffim set -- using the c2/compute3 group of nodes as ippmd in local pantasks. should run fine along w/ 3PI skycal

mark is czar

ps1 still down, no data last night
09:35 MEH: pstamp running for while, doing regular restart
16:40 MEH: looks like ipp071 has been down for 3ks -- new machine, not in service yet

mark is czar

no data last night
10:00 MEH: possibly data tonight, doing regular restart of pantasks in preparation
16:50 MEH: large set of biases downloading
20:00 MEH: yay data
23:00 MEH: downloads seem to be slipping (43 incomp) -- ipp033-036 still transferring data?

morning Bill: started up distribution of the LAP skycal runs. Needed to change labels for all but 100,000 to LAP.ThreePi.20130717.wait because having 900k+ runs to do made the query take a very long time.

06:01 Bill: first batch of skycal runs have finished distribution. Changed labels for another 200,000. The pending query takes up to 3 minutes with this setting.
- 06:05 The distribution pantasks is a little tired. pcontrol is spinning and runs are getting queued slowly. I'm going to restart and change the list of stages being distributed to only include skycal for now.
12:00 MEH: MOPS was missing some diffims, warp memory fault 4 cleared after a few manual reverts: o6864g0148o, 983502, skycell.1024.093

11:30 MEH: c2/compute3 group not fully utilized, running some stacks there as well

Note: See TracWiki for help on using the wiki.