IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:PS1_IPP_Czarlog_20130408

Version 21 (modified by Mark Huber, 13 years ago) ( diff )

--

PS1 IPP Czar Logs for the week 2013.04.08 - 2013.04.14

    1. PS1 IPP Czar Logs for the week 2013.04.08 - 2013.04.14
      1. Monday : 2013.04.08
      2. Tuesday : 2013.04.09
      3. Wednesday : 2013.04.10
      4. Thursday : 2013.04.11
      5. Friday : 2013.04.12
      6. Saturday : 2013.04.13
      7. Sunday : 2013.04.14

(Up to PS1 IPP Czar Logs)

Monday : 2013.04.08

  • 10:15 Bill stopped pstamp pantasks to perform mysqldump of ippc17 ippRequestServer. These have not been done regularty since ippc19 went off line. Will restart it once the backup finishes.
  • 14:15 Bill adjusted the MOPS postage stamp priority. QUB has submited over 10,000 requests which are going to block MOPS with the default settings

Tuesday : 2013.04.09

  • 06:10 Serge: ippdb02 backup on ippc63 and ipp064 complete. Restarting replication on ippdb02.
  • 10:25 Bill: update pantasks died. Started up a new one. Also restarted pstamp pantasks.
  • 12:10 CZW: stopped processing to allow Rita to fix a cabling issue on ipp020 and ipp021.
  • 12:23 haf: i'm cleaning up the faulted isp - we now have a script that finds the faulted ones, gets the proper bytes/md5sum, and changes the db.. we don't like this...
  • 13:20 CZW: processing restarted. ipp020 and ipp021 are back up.
  • 15:30 CZW: processing stopped for ippdb00 disk upgrade procedure.
  • 16:30 CZW: all clear from Serge. Restarting pantasks servers and starting processing (except cleanup, which will remain off).
  • 17:20 SC: restarted pstamp and update after ippc17 reboot
  • 20:00 SC: created ipp user on ippdb02. Should be useful for czartool
  • 23:20 Serge: All seems fine

Wednesday : 2013.04.10

  • 12:35 Bill The pstamp pantasks is running cleanup of old data which will cause the load on ippc17 to appear high. Scratch that wait until mops and qub pstamps are finished.

Thursday : 2013.04.11

  • 12:00 MEH: Gene put stsci06 into neb-host up state after rebooting into 3.7.6 kernel yesterday. MD08.pv1.20130403 running warps only to give it something to read/write (chip.off until nightly data starts unless needed)
  • 19:55 MEH: registration seems to be getting hung up easily lately, manually reverting problem get get nightly processing going

Friday : 2013.04.12

  • 20:10 MEH: looks like stdscience is due for regular restart. doing before nightly data gets going full force
  • 23:00 MEH: again download fault, manually revert to help along

Saturday : 2013.04.13

  • 10:50 MEH: nightly data finished, no problems with db02 and cleanup so adding MD processing back in

Sunday : 2013.04.14

Note: See TracWiki for help on using the wiki.