IPP Software Navigation Tools IPP Links Communication Pan-STARRS Links
wiki:Moped_accident

Version 11 (modified by Serge CHASTEL, 14 years ago) ( diff )

--

Just in case there is any problem and I can't do things myself

  • DONE change apache configuration files on ippc01-ippc10

At 8pm

  • DONE (scripts are finished): check if jt scripts are still running on ippc19: make them asleep
  • DONE make sure ippdb02 is synchronized with ippdb00, stop slave on ippb02
  • DONE apache is connected to ippdb02 (check process table) restart apache servers on ippc01-ippc10
  • N/A wake up jt scripts: if crashed, tell jt
  • DONE kill nebdiskd on ippdb00
  • RUNNING start nebulous mysqldump on ippdb00 (use pbzip2)

2012-05-29 19:00 The dump is running in screen session named nebulous_dump (i.e. reconnect on ippdb00 with screen -r nebulous_dump). The dump file is /export/ippdb00.0/nebulous_dumps/nebulous-20120529_fulldump.sql.bz2 and should have a size of 26G (according to the last successful dump of ippdb02 started this morning) 2012-05-29 19:50 The dump should be finished at 23:50.

Once neb dump is finished

  • copy reference dump to ippdb02 (and from ippdb02 to ipp001)
  • delete nebulous database on ippdb00
  • stop the mysql server
  • clean mysql on ippdb00 (remove old binlogs, move and compress mysql logs)
  • start the mysql server
  • ingest neb dump on ippdb00
  • write down new master coordinates
  • move and compress nebdiskd log
  • restart nebdiskd

Once ingestion is finished

  • restore apache configuration files on ippc01-ippc10 (backup files are called /etc/apache2/modules.d/apache2-mod_perl-startup.pl.20120529_ippdb00)
  • restart apache servers (jt scripts should be finished?!)
  • restart ipp
  • delete neb dump on ippdb00

Once everything seems fine, * on ippb02 *

  • comment neb dump in root crontab
  • delete nebulous database
  • stop the mysql server
  • clean mysql (remove old binlogs, move and compress mysql logs)
  • start the mysql server
  • ingest neb dump
  • set master coordinates
  • start slave
  • activate neb dump in root crontab
  • delete neb dump
Note: See TracWiki for help on using the wiki.