bucardo.git
11 years agoadd_sync: fix checking for existing dbgroup; add back in confirm_commit
Greg Sabino Mullane [Sat, 7 Jun 2014 19:36:55 +0000 (15:36 -0400)]
add_sync: fix checking for existing dbgroup; add back in confirm_commit

11 years agoMake tables=all (e.g. for add_sync) add only tables with pkeys or unique indexes...
Greg Sabino Mullane [Sat, 7 Jun 2014 19:32:14 +0000 (15:32 -0400)]
Make tables=all (e.g. for add_sync) add only tables with pkeys or unique indexes; also excludes the bucardo schema.

11 years agoUse savepoints when removing databases, in case we need to use the evals
Greg Sabino Mullane [Sat, 7 Jun 2014 18:58:55 +0000 (14:58 -0400)]
Use savepoints when removing databases, in case we need to use the evals

11 years agoAdd new ENV BUCARDO_CONFIRM which sets bcargs->{confirm} inside of the bucardo program.
Greg Sabino Mullane [Sat, 7 Jun 2014 18:43:20 +0000 (14:43 -0400)]
Add new ENV  BUCARDO_CONFIRM which sets bcargs->{confirm} inside of the bucardo program.

11 years agoMild doc updates
Greg Sabino Mullane [Sat, 7 Jun 2014 18:37:10 +0000 (14:37 -0400)]
Mild doc updates

11 years agoUse smallint, integer, and bigint rather than int[248] for the column types in the...
Greg Sabino Mullane [Sat, 7 Jun 2014 18:29:27 +0000 (14:29 -0400)]
Use smallint, integer, and bigint rather than int[248] for the column types in the goat table.

11 years agoAllow add_database to take more than one arg so we can add multiple databases at...
Greg Sabino Mullane [Sat, 7 Jun 2014 15:10:44 +0000 (11:10 -0400)]
Allow add_database to take more than one arg so we can add multiple databases at once via command line

11 years agoAllow the 'dbs' argument to specify a dbgroup when doing "add_sync", for backwards...
Greg Sabino Mullane [Fri, 6 Jun 2014 22:32:12 +0000 (18:32 -0400)]
Allow the 'dbs' argument to specify a dbgroup when doing "add_sync", for backwards compatibility.

11 years agoRemove few instances of trailing whitespace
Greg Sabino Mullane [Fri, 6 Jun 2014 19:51:11 +0000 (15:51 -0400)]
Remove few instances of trailing whitespace

11 years agoTrailing whitespace check for Bucardo.pm too
Greg Sabino Mullane [Fri, 6 Jun 2014 19:50:56 +0000 (15:50 -0400)]
Trailing whitespace check for Bucardo.pm too

11 years agoQuick trailing whitespace testing
Greg Sabino Mullane [Fri, 6 Jun 2014 19:47:59 +0000 (15:47 -0400)]
Quick trailing whitespace testing

11 years agoMake add_sync much smarter and clearer. Should now be able to add dbgroups, dbs,...
Greg Sabino Mullane [Fri, 6 Jun 2014 19:47:21 +0000 (15:47 -0400)]
Make add_sync much smarter and clearer. Should now be able to add dbgroups, dbs, tables, and relgroups more intuitively.

11 years agoUse -l to check for a symlink.
David E. Wheeler [Tue, 3 Jun 2014 21:22:36 +0000 (14:22 -0700)]
Use -l to check for a symlink.

Not -s. From [-X.pod](http://perldoc.perl.org/functions/-X.html):

    -s  File has nonzero size (returns size in bytes).
    -l  File is a symbolic link.

Bug introduced in 9a1c0dbf.

11 years agoAllow args to use : or = as delimiters.
Greg Sabino Mullane [Sun, 1 Jun 2014 22:07:48 +0000 (18:07 -0400)]
Allow args to use : or = as delimiters.
Allow more aliases for tables in add sync calls.

11 years agoSet the log prefix immediately after forking, based on the passed-in type
Greg Sabino Mullane [Sun, 1 Jun 2014 20:34:09 +0000 (16:34 -0400)]
Set the log prefix immediately after forking, based on the passed-in type

11 years agoSpellcheck and linty cleanups
Greg Sabino Mullane [Sat, 31 May 2014 18:38:43 +0000 (14:38 -0400)]
Spellcheck and linty cleanups

11 years agoMake sure we import our methods!
Greg Sabino Mullane [Sat, 31 May 2014 12:34:54 +0000 (08:34 -0400)]
Make sure we import our methods!

11 years agoTabs to spaces
Greg Sabino Mullane [Sat, 31 May 2014 12:30:07 +0000 (08:30 -0400)]
Tabs to spaces

11 years agoDo not like Encode::Locale dependency, but since it is here, might as well use its...
Greg Sabino Mullane [Sat, 31 May 2014 12:28:55 +0000 (08:28 -0400)]
Do not like Encode::Locale dependency, but since it is here, might as well use its method for argv decoding.

11 years agoChange wording in examples as we use target, not slave. Not even sure the commented...
Greg Sabino Mullane [Thu, 29 May 2014 18:26:38 +0000 (14:26 -0400)]
Change wording in examples as we use target, not slave. Not even sure the commented examples in bucardo are valid anymore!

11 years agoTweak some redo logic
Greg Sabino Mullane [Wed, 28 May 2014 20:30:58 +0000 (16:30 -0400)]
Tweak some redo logic

11 years agoWe need target column on index to be text_pattern_ops, as we are going to start looki...
Greg Sabino Mullane [Sun, 25 May 2014 20:17:24 +0000 (16:17 -0400)]
We need target column on index to be text_pattern_ops, as we are going to start looking for partials.

11 years agoSomeone fix this test, please.
Greg Sabino Mullane [Sun, 25 May 2014 19:58:30 +0000 (15:58 -0400)]
Someone fix this test, please.

11 years agoNo more ^T: in the main distinct delta gathering clause: there will be a new way...
Greg Sabino Mullane [Sat, 24 May 2014 16:46:41 +0000 (12:46 -0400)]
No more ^T: in the main distinct delta gathering clause: there will be a new way to do things, linked to the new partial sync system.

11 years agoFirst swing at handling dead databases in a smooth manner.
Greg Sabino Mullane [Fri, 23 May 2014 14:40:05 +0000 (10:40 -0400)]
First swing at handling dead databases in a smooth manner.
If a database is unreachable, we mark it as stalled. The sync containing it is marked as stalled as well.
This should allow other syncs to continue even if databases from another sync are down.
We also re-check stalled databases on startup and restore things to an active state if we can.
Still todo:
* tests
* handle other dbi databases
* handle non-dbi databases
* have mcp periodically check if we can restore stalled dbs/syncs.
* cleanup/refactor the code

11 years agoDo a check for a stopfile after MCP sleeps. Prevents infinite loop if something goes...
Greg Sabino Mullane [Thu, 22 May 2014 16:00:29 +0000 (12:00 -0400)]
Do a check for a stopfile after MCP sleeps. Prevents infinite loop if something goes wrong before the MCP can spin up all the way.

11 years agoMinor fixes
Greg Sabino Mullane [Tue, 20 May 2014 15:11:32 +0000 (11:11 -0400)]
Minor fixes

11 years agoAdd support for "bucardo reopen" command, which will cause all processes to re-open...
Greg Sabino Mullane [Mon, 19 May 2014 17:38:49 +0000 (13:38 -0400)]
Add support for "bucardo reopen" command, which will cause all processes to re-open any existing file handles.
Useful for log rotations.

11 years agoFramework for new 'reopen' command
Greg Sabino Mullane [Fri, 16 May 2014 16:13:35 +0000 (12:13 -0400)]
Framework for new 'reopen' command

11 years agoPrint the last git commit information on startup
Greg Sabino Mullane [Wed, 7 May 2014 16:58:37 +0000 (12:58 -0400)]
Print the last git commit information on startup

11 years agoDon't try to use pg_cancel if things blew up, simply rollback and set async_active...
Greg Sabino Mullane [Wed, 7 May 2014 15:50:41 +0000 (11:50 -0400)]
Don't try to use pg_cancel if things blew up, simply rollback and set async_active off!

11 years agoMake sure we only update bucardo_truncate_trigger if we really need to, per report...
Greg Sabino Mullane [Mon, 5 May 2014 14:29:03 +0000 (10:29 -0400)]
Make sure we only update bucardo_truncate_trigger if we really need to, per report of severe bloat on the mailing list.

11 years agoPrevent uninitialized warnings from no db.service on validate, per mailing list compl...
Greg Sabino Mullane [Mon, 5 May 2014 14:24:42 +0000 (10:24 -0400)]
Prevent uninitialized warnings from no db.service on validate, per mailing list complaint.

11 years agoRename the somewhat confusing bucardo version configuration variables.
Greg Sabino Mullane [Mon, 5 May 2014 14:19:46 +0000 (10:19 -0400)]
Rename the somewhat confusing bucardo version configuration variables.
bucardo_version becomes bucardo_initial_version
bucardo_current_version becomes bucardo_version

11 years agoMild TODO cleanup
Greg Sabino Mullane [Mon, 5 May 2014 14:19:27 +0000 (10:19 -0400)]
Mild TODO cleanup

11 years agoNo sense it sending notification if the db is down
Greg Sabino Mullane [Sun, 4 May 2014 14:24:53 +0000 (10:24 -0400)]
No sense it sending notification if the db is down

11 years agoMore tweaking to async code
Greg Sabino Mullane [Fri, 2 May 2014 23:34:44 +0000 (19:34 -0400)]
More tweaking to async code

11 years agoRandom comments about new plan
Greg Sabino Mullane [Fri, 2 May 2014 22:07:35 +0000 (18:07 -0400)]
Random comments about new plan

11 years agoFix typo
Greg Sabino Mullane [Wed, 30 Apr 2014 11:27:41 +0000 (07:27 -0400)]
Fix typo

11 years agoDebugging to solve pg_result errors.
Greg Sabino Mullane [Tue, 29 Apr 2014 22:39:42 +0000 (18:39 -0400)]
Debugging to solve pg_result errors.

11 years agoGrabbed incorrect hash - was causing the async_status to not get checked and cleared...
Greg Sabino Mullane [Tue, 29 Apr 2014 18:10:30 +0000 (14:10 -0400)]
Grabbed incorrect hash - was causing the async_status to not get checked and cleared properly on serialization failures occuring at the pg_result point. Shoudl clear up the recent problem seen on the mailing list.

11 years agoSig for 4.99.12
Greg Sabino Mullane [Fri, 25 Apr 2014 17:25:07 +0000 (13:25 -0400)]
Sig for 4.99.12

11 years agoBump version to 4.99.12 4.99.12
Greg Sabino Mullane [Fri, 25 Apr 2014 17:23:10 +0000 (13:23 -0400)]
Bump version to 4.99.12

11 years agoBetter tracking of async status
Greg Sabino Mullane [Thu, 24 Apr 2014 00:07:04 +0000 (20:07 -0400)]
Better tracking of async status

11 years agoBetter service arg parsing
Greg Sabino Mullane [Sun, 20 Apr 2014 12:23:59 +0000 (08:23 -0400)]
Better service arg parsing

11 years agoreadlink can sometimes retrurn undef, even if -s is true, so make a safety check
Greg Sabino Mullane [Sun, 20 Apr 2014 11:29:22 +0000 (07:29 -0400)]
readlink can sometimes retrurn undef, even if -s is true, so make a safety check

11 years agoMild tweaks to log_level notifications
Greg Sabino Mullane [Tue, 15 Apr 2014 15:02:23 +0000 (11:02 -0400)]
Mild tweaks to log_level notifications

11 years agoSupport symlinked bucardo executable when looking for Bucardo.pm and bucardo.schema
Greg Sabino Mullane [Tue, 15 Apr 2014 15:01:24 +0000 (11:01 -0400)]
Support symlinked bucardo executable when looking for Bucardo.pm and bucardo.schema
Allow use of "all" and "tables=all" when creating a herd/relgroup
Don't keep retrying connection if in batch mode

11 years agosqlite: do a BEGIN EXCLUSIVE TRANSACTION in force-lock mode.
Greg Sabino Mullane [Sat, 12 Apr 2014 18:33:50 +0000 (14:33 -0400)]
sqlite: do a BEGIN EXCLUSIVE TRANSACTION in force-lock mode.

11 years agoPut the force-lock-file in the piddir, rather than the globally writeable /tmp
Greg Sabino Mullane [Sat, 12 Apr 2014 18:25:42 +0000 (14:25 -0400)]
Put the force-lock-file in the piddir, rather than the globally writeable /tmp

11 years agoTake out the session level isolation level for mysql - our per transaction calls...
Greg Sabino Mullane [Sat, 12 Apr 2014 17:58:52 +0000 (13:58 -0400)]
Take out the session level isolation level for mysql - our per transaction calls should be good enough.

11 years agoUse READ WRITE isolation levels wherever we can.
Greg Sabino Mullane [Sat, 12 Apr 2014 17:50:57 +0000 (13:50 -0400)]
Use READ WRITE isolation levels wherever we can.

11 years agoHave kid_pid_start notification happen on the main channel, not per database
Greg Sabino Mullane [Sat, 12 Apr 2014 17:31:05 +0000 (13:31 -0400)]
Have kid_pid_start notification happen on the main channel, not per database

11 years agoGather the maximum transaction time for all tables in a single UNION query,
Greg Sabino Mullane [Sat, 12 Apr 2014 17:22:55 +0000 (13:22 -0400)]
Gather the maximum transaction time for all tables in a single UNION query,
rather than making separate SQL calls for each table.
Per suggestion on mailing list from Hans van der Riet

11 years agoAdd error NOTIFY when reload fails
Joshua Tolley [Thu, 10 Apr 2014 17:18:42 +0000 (11:18 -0600)]
Add error NOTIFY when reload fails

11 years agoExit right away if the delta test fails, for better debugging.
Greg Sabino Mullane [Fri, 11 Apr 2014 17:28:17 +0000 (13:28 -0400)]
Exit right away if the delta test fails, for better debugging.

11 years agoWhen debug is on, write out a small text file to help connect to the test databases...
Greg Sabino Mullane [Fri, 11 Apr 2014 17:26:04 +0000 (13:26 -0400)]
When debug is on, write out a small text file to help connect to the test databases via psql.

11 years agoClean up confusing comment
Greg Sabino Mullane [Fri, 11 Apr 2014 11:39:08 +0000 (07:39 -0400)]
Clean up confusing comment

11 years agoCleanup the temporary service files
Greg Sabino Mullane [Fri, 11 Apr 2014 11:38:45 +0000 (07:38 -0400)]
Cleanup the temporary service files

11 years agoMove kid_pid_stop to global listening, not db-based.
Greg Sabino Mullane [Thu, 10 Apr 2014 22:36:46 +0000 (18:36 -0400)]
Move kid_pid_stop to global listening, not db-based.
Do a ping check before trying to cleanup databases on the kid exception handler.

11 years agoBetter debugging for notifys
Greg Sabino Mullane [Thu, 10 Apr 2014 18:25:42 +0000 (14:25 -0400)]
Better debugging for notifys

11 years agoRewrite the delete_rows function.
Greg Sabino Mullane [Thu, 10 Apr 2014 18:00:08 +0000 (14:00 -0400)]
Rewrite the delete_rows function.
Loop through and make sure we have the maximum number of async targets running at all times.
Moved much of the logic to per-target attribs.

11 years agoDebug the initdb call during testing
Greg Sabino Mullane [Thu, 10 Apr 2014 13:43:28 +0000 (09:43 -0400)]
Debug the initdb call during testing

11 years agoBetter tweaking for pgservice items
Greg Sabino Mullane [Thu, 10 Apr 2014 13:42:46 +0000 (09:42 -0400)]
Better tweaking for pgservice items

11 years agoRemove debugging.
Greg Sabino Mullane [Wed, 9 Apr 2014 23:20:08 +0000 (19:20 -0400)]
Remove debugging.

11 years agoBetter munging if we are using a service
Greg Sabino Mullane [Wed, 9 Apr 2014 23:18:20 +0000 (19:18 -0400)]
Better munging if we are using a service

11 years agoIgnore any local .bucardorc files.
Greg Sabino Mullane [Sun, 6 Apr 2014 20:07:03 +0000 (16:07 -0400)]
Ignore any local .bucardorc files.

11 years agoRemove useless comment
Greg Sabino Mullane [Tue, 1 Apr 2014 11:05:29 +0000 (07:05 -0400)]
Remove useless comment

11 years agoBump copyright to 2014
Greg Sabino Mullane [Mon, 31 Mar 2014 13:27:17 +0000 (09:27 -0400)]
Bump copyright to 2014

11 years agoAdd test for 'bucardo upgrade'
Joshua Tolley [Thu, 27 Mar 2014 21:01:09 +0000 (15:01 -0600)]
Add test for 'bucardo upgrade'

11 years agoMore bucardo.upgrade fixes
Joshua Tolley [Thu, 27 Mar 2014 20:51:47 +0000 (14:51 -0600)]
More bucardo.upgrade fixes

Now bucardo upgrade run against a database that doesn't need any changes
correctly reports that no changes are necessary

11 years agoIf we are using a service file, make sure we don't default slurp in host and port...
Greg Sabino Mullane [Thu, 20 Mar 2014 21:08:51 +0000 (17:08 -0400)]
If we are using a service file, make sure we don't default slurp in host and port env variables

11 years agogit should ignore service testing files
Greg Sabino Mullane [Thu, 20 Mar 2014 18:24:35 +0000 (14:24 -0400)]
git should ignore service testing files

11 years agoModify syntax so the upgrade process can understand it
Joshua Tolley [Thu, 20 Mar 2014 17:36:27 +0000 (11:36 -0600)]
Modify syntax so the upgrade process can understand it

11 years agoMake the dbservice column have an empty default, so we don't have to start checking...
Greg Sabino Mullane [Mon, 17 Mar 2014 19:38:49 +0000 (15:38 -0400)]
Make the dbservice column have an empty default, so we don't have to start checking "defined or length..." everywhere

11 years agoRevert comment, so it stays in sync with reverted code
Joshua Tolley [Wed, 12 Mar 2014 18:29:44 +0000 (12:29 -0600)]
Revert comment, so it stays in sync with reverted code

11 years agoFinish supporting non-ASCII table and column names
Joshua Tolley [Wed, 12 Mar 2014 16:30:47 +0000 (10:30 -0600)]
Finish supporting non-ASCII table and column names

11 years agoBucardo now accepts pg_service for databases
Joshua Tolley [Tue, 18 Feb 2014 20:06:37 +0000 (13:06 -0700)]
Bucardo now accepts pg_service for databases

11 years agoservice strings are working; still needs tests
Joshua Tolley [Fri, 14 Feb 2014 17:59:34 +0000 (10:59 -0700)]
service strings are working; still needs tests

11 years agoAlways use async when doing deletes. The previous way had a thinko - this should...
Greg Sabino Mullane [Tue, 18 Feb 2014 14:54:46 +0000 (09:54 -0500)]
Always use async when doing deletes. The previous way had a thinko - this should ensure large deletes run concurrently on different hosts.

11 years agoMake install quit ignoring dbname and dbuser options
Joshua Tolley [Thu, 13 Feb 2014 23:25:17 +0000 (16:25 -0700)]
Make install quit ignoring dbname and dbuser options

... again.

11 years agoDon't assume the bucardo database exists on install
Joshua Tolley [Fri, 14 Feb 2014 19:59:48 +0000 (12:59 -0700)]
Don't assume the bucardo database exists on install

11 years agoRevert "Make install quit ignoring dbname and dbuser options"
Joshua Tolley [Fri, 14 Feb 2014 18:14:18 +0000 (11:14 -0700)]
Revert "Make install quit ignoring dbname and dbuser options"

I should probably have run the test suite before committing...

This reverts commit 39dcde241abcdfcc45415d2eb210d8b2c694f479.

11 years agoMake install quit ignoring dbname and dbuser options
Joshua Tolley [Thu, 13 Feb 2014 23:25:17 +0000 (16:25 -0700)]
Make install quit ignoring dbname and dbuser options

11 years agos/vaccuum/vacuum/
Joshua Tolley [Thu, 13 Feb 2014 20:55:15 +0000 (13:55 -0700)]
s/vaccuum/vacuum/

11 years agoMake our statement_chunk_size default match up.
Greg Sabino Mullane [Fri, 17 Jan 2014 15:49:09 +0000 (10:49 -0500)]
Make our statement_chunk_size default match up.

11 years agoDebugging code
Greg Sabino Mullane [Thu, 16 Jan 2014 15:58:40 +0000 (10:58 -0500)]
Debugging code

11 years agoPassword should default to undef, not an empty string
Greg Sabino Mullane [Thu, 16 Jan 2014 15:42:02 +0000 (10:42 -0500)]
Password should default to undef, not an empty string

11 years agoGotta have log levels on all glog() calls.
Greg Sabino Mullane [Sun, 22 Dec 2013 16:12:52 +0000 (11:12 -0500)]
Gotta have log levels on all glog() calls.

11 years agoVersion bump to 4.99.11
Greg Sabino Mullane [Sun, 22 Dec 2013 16:12:15 +0000 (11:12 -0500)]
Version bump to 4.99.11

11 years agoEval everything.
David E. Wheeler [Tue, 17 Dec 2013 19:46:21 +0000 (11:46 -0800)]
Eval everything.

11 years agoSomeone cannot count.
David E. Wheeler [Tue, 17 Dec 2013 01:11:53 +0000 (17:11 -0800)]
Someone cannot count.

11 years agoFix typo in comment.
David E. Wheeler [Tue, 17 Dec 2013 01:00:14 +0000 (17:00 -0800)]
Fix typo in comment.

11 years agoCheck @tableoids, not tablelist, as the latter may be empty (inside validate_sync)
Greg Sabino Mullane [Tue, 3 Dec 2013 17:10:55 +0000 (12:10 -0500)]
Check @tableoids, not tablelist, as the latter may be empty (inside validate_sync)

11 years agoMeta things for the new dev/ directory
Greg Sabino Mullane [Sat, 30 Nov 2013 16:43:45 +0000 (11:43 -0500)]
Meta things for the new dev/ directory

11 years agoVarious tests for developers
Greg Sabino Mullane [Sat, 30 Nov 2013 16:42:15 +0000 (11:42 -0500)]
Various tests for developers

11 years agoBetter makedelta test. Need to move these somewhere else.
Greg Sabino Mullane [Sat, 30 Nov 2013 16:41:06 +0000 (11:41 -0500)]
Better makedelta test. Need to move these somewhere else.

11 years agoFix up some of the makedelta logic.
Greg Sabino Mullane [Sat, 30 Nov 2013 16:39:34 +0000 (11:39 -0500)]
Fix up some of the makedelta logic.

11 years agoBetter handling of inactive databases.
Greg Sabino Mullane [Fri, 22 Nov 2013 04:24:03 +0000 (23:24 -0500)]
Better handling of inactive databases.

11 years agoAdd a fifth inactive database to ensure we handle inactive targets gracefully.
Greg Sabino Mullane [Fri, 22 Nov 2013 03:20:52 +0000 (22:20 -0500)]
Add a fifth inactive database to ensure we handle inactive targets gracefully.