walmgr.py <config.ini> command
-== Step-by-step instructions ==
+== DESCRIPTION ==
-1. Set up no-password ssh access from one to other
+aaa
+
+== QUICK START ==
+
+1. Set up passwordless ssh authentication from master to slave
master$ test -f ~/.ssh/id_dsa.pub || ssh-keygen -t dsa
- master$ scp .ssh/id_dsa.pub slave:
- slave$ cat id_dsa.pub >> ~/.ssh/authorized_keys
+ master$ cat ~/.ssh/id_dsa.pub | ssh slave cat \>\> .ssh/authorized_keys
2. Configure paths
master$ edit master.ini
slave$ edit slave.ini
- slave$ mkdir data.master logs.full logs.partial
-3. Start archival process
+ Make sure that walmgr.py executable has same pathname on slave and master.
+
+3. Start archival process and create a base backup
master$ ./walmgr.py master.ini setup
+ master$ ./walmgr.py master.ini backup
-4. Do full backup+restore
+4. Prepare postgresql.conf and pg_hba.conf on slave and start replay
- master$ ./walmgr.py master.ini backup
- slave$ ./walmgr.py slave.ini restore
+ master$ scp $PGDATA/*.conf slave:
+ slave$ ./walmgr.py slave.ini restore
- 'walmgr.py restore' moves data in place and starts postmaster,
- that starts replaying logs as they appear.
+ For debian based distributions the standard configuration files are located
+ in /etc/postgresql/x.x/main directory. If another scheme is used the postgresql.conf
+ and pg_hba.conf should be copied to slave full_backup directory. Make sure to
+ disable archive_command in slave config.
+
+ 'walmgr.py restore' moves data in place, creates recovery.conf and starts postmaster
+ in recovery mode.
5. In-progress WAL segments can be backup by command:
slave$ ./walmgr.py slave.ini boot
+== GENERAL OPTIONS ==
+
+Common options to all walmgr.py commands.
+
+ -h, --help::
+ show this help message and exit
+
+ -q, --quiet::
+ make program silent
+
+ -v, --verbose::
+ make program more verbose
+
+ -n, --not-really::
+ Show what would be done without actually doing anything.
+
+== MASTER COMMANDS ==
+
+=== setup ===
+Sets up postgres archiving, creates necessary directory structures on slave.
+
+=== sync ===
+Synchronizes in-progress WAL files to slave.
+
+=== syncdaemon ===
+Start WAL synchronization in daemon mode. This will start periodically synching
+the in-progress WAL files to slave.
+
+loop_delay
+use_xlog_functions
+
+=== stop ===
+Deconfigures postgres archiving.
+
+=== periodic ===
+Runs periodic command, if configured. This enables to execute arbitrary commands
+on interval, useful for synchronizing scripts, config files, crontabs etc.
+
+=== listbackups ===
+List backup sets available on slave node.
+
+=== backup ===
+Creates a new base backup from master database. Will purge expired backups and WAL
+files on slave if `keep_backups` is specified. During a backup a lock file is
+created in slave `completed_wals` directory. This is to prevent simultaneous
+backups and resulting corruption. If running backup is terminated, the BACKUPLOCK
+file may have to be removed manually.
+
+=== restore <set> <dst> ===
+EXPERIMENTAL. Attempts to restore the backup from slave to master.
+
+== SLAVE COMMANDS ==
+
+=== boot ===
+Stop log playback and bring the database up.
+
+=== pause ===
+Pauses WAL playback.
+
+=== continue ===
+Continues previously paused WAL playback.
+
+=== listbackups ===
+Lists available backups.
+
+=== backup ===
+EXPERIMENTAL. Creates a new backup from slave data. Log replay is paused,
+slave data directory is backed up to `full_backup` directory and log
+replay resumed. Backups are rotated as needed. The idea is to move the
+backup load away from production node. Usable from postgres 8.2 and up.
+
+=== restore [src][dst] ===
+Restores the specified backup set to target directory. If specified without
+arguments the latest backup is *moved* to slave data directory (doesn't obey
+retention rules). If src backup is specified the backup is copied (instead of moving).
+Alternative destination directory can be specified with `dst`.
+
== CONFIGURATION ==
-=== master.ini ===
+=== Common settings ===
+
+==== job_name ====
+Optional. Indentifies this script, used in logging. Keep unique if
+using central logging.
+
+==== logfile ====
+Where to log.
+
+==== use_skylog ====
+Optional. If nonzero, skylog.ini is used for log configuration.
+
+=== Master settings ===
+
+==== pidfile ====
+Pid file location for syncdaemon mode (if running with -d). Otherwise
+not required.
+
+==== master_db ====
+Database to connect to for pg_start_backup() etc. It is not a
+good idea to use `dbname=template` if running syncdaemon in
+record shipping mode.
+
+==== master_data ====
+Master data directory location.
+
+==== master_config ====
+Master postgresql.conf file location. This is where
+`archive_command` gets updated.
+
+==== slave ====
+Slave host and base directory.
+
+==== completed_wals ====
+Slave directory where archived WAL files are copied.
+
+==== partial_wals ====
+Slave directory where incomplete WAL files are stored.
+
+==== full_backup ====
+Slave directory where full backups are stored.
+
+==== loop_delay ====
+The frequency of syncdaemon updates. In record shipping mode only
+incremental updates are sent, so smaller interval can be used.
+
+==== use_xlog_functions ====
+Use pg_xlog functions for record based shipping (available in 8.2 and up).
+
+==== compression ====
+If nonzero, a -z flag is added to rsync cmdline. Will reduce network
+traffic at the cost of extra CPU time.
+
+==== periodic_command ====
+Shell script to be executed at specified time interval. Can be used for
+synchronizing scripts, config files etc.
+
+==== command_interval ====
+How ofter to run periodic command script. In seconds, and only evaluated
+at log switch times.
+
+=== Sample master.ini ===
[wal-master]
logfile = master.log
- use_skylog = 0
-
+ pidfile = master.pid
master_db = dbname=template1
master_data = /var/lib/postgresql/8.0/main
master_config = /etc/postgresql/8.0/main/postgresql.conf
-
- slave = slave:/var/lib/postgresql/walshipping
-
+ slave = slave:/var/lib/postgresql/walshipping
completed_wals = %(slave)s/logs.complete
partial_wals = %(slave)s/logs.partial
full_backup = %(slave)s/data.master
-
- # syncdaemon update frequency
loop_delay = 10.0
+ use_xlog_functions = 1
+ compression = 1
-=== slave.ini ===
+=== Slave settings ===
+
+==== slave_data ====
+Postgres data directory for the slave. This is where the restored
+backup is copied/moved.
+
+==== slave_stop_cmd ====
+Script to stop postmaster on slave.
+
+==== slave_start_cmd ====
+Script to start postmaster on slave.
+
+==== slave ====
+Base directory for slave files (logs.comlete, data.master etc)
+
+==== completed_wals ====
+Directory where complete WAL files are stored. Also miscellaneous control files
+are created in this directory (BACKUPLOCK, STOP, PAUSE, etc.).
+
+==== partial_wals ====
+Directory where partial WAL files are stored.
+
+==== full_backup ====
+Directory where full backups are stored.
+
+==== keep_backups ====
+Number of backups to keep. Also all WAL files needed to bring earliest
+
+backup up to date are kept. The backups are rotated before new backup
+is started, so at one point there is actually one less backup available.
+
+It probably doesn't make sense to specify `keep_backups` if periodic
+backups are not performed - the WAL files will pile up quickly.
+
+Backups will be named data.master, data.master.0, data.master.1 etc.
+
+==== archive_command ====
+Script to execute before rotating away the oldest backup. If it fails
+backups will not be rotated.
+
+=== Sample slave.ini ===
[wal-slave]
logfile = slave.log
- use_skylog = 0
-
slave_data = /var/lib/postgresql/8.0/main
slave_stop_cmd = /etc/init.d/postgresql-8.0 stop
slave_start_cmd = /etc/init.d/postgresql-8.0 start
-
- slave = /var/lib/postgresql/walshipping
+ slave = /var/lib/postgresql/walshipping
completed_wals = %(slave)s/logs.complete
partial_wals = %(slave)s/logs.partial
full_backup = %(slave)s/data.master
-
- keep_old_logs = 0
+ keep_backups = 5
projects / skytools.git / commitdiff