PS1 IPP Czar Logs for the week 2011.04.18 - 2011.04.25

(Up to PS1 IPP Czar Logs)

Monday : 2011.04.18

• 10:36 Bill queued stacks for STS.refstack.20110418
• 11:08 CZW: Noticed that burntool crashed last night, leaving an exposure in state check_burntool instead of properly processing. Reset state to pending_burntool, and registration started up again to finish. Registration logfile (neb://ipp047.0/gpc1/20110418/o5669g0493o.326098/o5669g0493o.326098.reg.ota67.log) suggests a database error.
• 11:16 Bill has queued 111 STS exposures for chip-warp processing.
• 13:27 CZW: Accidentally ran pantasks with doomsday-debug stack information enabled. Trying to kill them all now.
• 14:43 Bill set STS.2010.a label to inactive to give last night's warps more power to finish.
• 16:30 Bill set STS.2010.a to active.
• 20:48 Bill restarted update pantasks because the large number of timeouts was computing the status listing.

Tuesday : 2011-04-19

Serge is czar (but there were no observation last night)

Wednesday : 2011-04-20

Serge is czar

• 13:55 distribution seemed sluggish so Bill restarted it.
• 14:18 CZW: concern about diskspace prompted me to look at the replication pantasks to see what shuffle was doing. The replication pantasks was not doing anything, because it failed to load nebulous.site.pro as part of the setup macro. It appears that this file was not transferred over between tag changes, so I've copied the version from /data/ippc18.0/home/ipp/psconfig/ipp-20110218.lin64/share/pantasks/modules into the working tag. This has unstuck the shuffle task.

Thursday : YYYY.MM.DD

• 02:06 CZW: noticed registration was lagging. register_imfile complained about a read only filesystem: neb://ipp005.0/gpc1/20110421/o5672g0156o/o5672g0156o.ota02.burn.log . Resetting the data_state for that imfile seems to have cleared subsequent exposures, but o5672g0156o seems to be stuck and unable to register the exposure correctly. I'll look at it tomorrow.

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD