(Up to PS1 IPP Czar Logs)

Monday : 2012.11.12

Mark is czar

• 08:00 MEH: last of nightly science finishing up. LAP and other reprocessing chip faults from permissions still.
  • chip.off for a while to push more LAP warps
• 11:30 still babysitting a large number of OSS diffims slowly making their way through..
• 14:30 OSS and 3PI diffims finished, regular restart of stdscience done.
• 21:00 MEH: poor weather so pushing more MD05 reprocessing through
• 21:30 repaired lost files, replaced with found orphans

  neb://ipp027.0/gpc1/20101102/o5502g0217o/o5502g0217o.ota26.fits
  neb://ipp051.0/gpc1/20100605/o5352g0039o/o5352g0039o.ota64.fits
  

Tuesday : 2012-11-13

Serge is czar

• 06:15: No observation last night. However ipp001 mysql server crashed: Nothing in its log though. Replication is only 10 minutes behind so it should not become a major incident.
• 14:00: Stopped all processing
• 14:45: Restarting all rpc.statd on all storage nodes:

  schastel@ippc11:~$ echo $PYTHONPATH 
  /home/panstarrs/schastel/local/lib64/python2.5/site-packages:/home/panstarrs/schastel/local/lib/python2.5/site-packages:/home/panstarrs/schastel/local/share/python
  schastel@ippc11:~/dev/SshSudo$ ./ssh_sudo.py -H `cat tmp/storageNodes` 'sudo /etc/init.d/rpc.statd restart'
  

• 15:12 (Bill) Since there are many postage stamp requests requiring update processing I added some more hosts to the update pantasks (hosts.update) and tweaked the polling variables: set.poll 80 in update and in pstamp set.dependent.poll 120
• 17:00: Chris queued a bunch of cleanup tasks.

Wednesday : 2012-11-14

Serge is czar

• 08:00: 35(?) science exposures last night. No observation report though.

Thursday : YYYY.MM.DD

Friday : YYYY.MM.DD

Saturday : YYYY.MM.DD

Sunday : YYYY.MM.DD