PS1_IPP_Czarlog_20130121 – Pan-STARRS IPP

Context Navigation

Version 14 (modified by Mark Huber, 13 years ago) ( diff )
--

12:02 Bill reloaded the module survey.pro into stdscience with the survey.skycal set to add -ra_min 120 to the command line. This is the area that we have most interest in. Staticsky has finished the region between 120 - 180 including skycells with < 5 filters. The area from 180 - 210 should finish later today. See first plot at http://svn.pan-starrs.ifa.hawaii.edu/trac/ipp/wiki/staticsky.20120706
16:38 Bill restarted stdscience

08:05 Bill added label goto_cleaned.rerun into the cleanup pantasks. There are 19711 chip runs set to get their cmf files removed.
10:41 Bill removed label goto_cleaned.rerun watching it's effect on staticsky completion rate. (That wasn't it set back on a couple of hours later)
14:56 Bill set stsci06 to repair mode
15:53 Bill set stsci06 back to up.
15:59 Bill set cleanup pantasks to stop. It's ready for a restart.
16:42 started cleanup pantasks

12:02 Bill has been investigating the reduction in staticsky throughput during the day. It's 3x faster at night when nightly science is also doing work. We believe that the culprit is stsci06. Set one set of wave3 nodes to off in deepstack to reduce the load.
13:05 Bill: Rita is going to stop the raid consistency check on stsci06. The I/O lag seems to not be happening. Set wave3 nodes back to on.
14:10 MEH: stsci00 disks put into neb-host repair in December due to 5% space buffer not being tracked by nebulous and could've resulted in badness. Gavin will look into setting down to 1% like other stsci disks after ipp027-ipp030 updates finished (data on stsci00), but there is +2TB free on the disks now so setting to up and will keep eye on. If goes red, need to set repair again.
15:09 Bill: stsci06 results inconclusive as yet. It does seem to show up less often on ganglia as high load
15:10 restarted pstamp and update pantasks. Their pcontrols were hogging cpu

Note: See TracWiki for help on using the wiki.