skytools.git
14 years agoMerge remote branch 'intgit/master'
Marko Kreen [Tue, 21 Dec 2010 12:39:47 +0000 (14:39 +0200)]
Merge remote branch 'intgit/master'

14 years agolambda vs quite_literal
Asko Tiidumaa [Tue, 21 Dec 2010 12:37:52 +0000 (12:37 +0000)]
lambda vs quite_literal

14 years agofixed local_only functionality, structural changes
Asko Tiidumaa [Mon, 20 Dec 2010 18:22:48 +0000 (18:22 +0000)]
fixed local_only functionality, structural changes

14 years agoadded local_only to londiste config parameter - accepts only events for local tables
Asko Tiidumaa [Mon, 20 Dec 2010 16:57:38 +0000 (16:57 +0000)]
added local_only to londiste config parameter - accepts only events for local tables

14 years agoqaadmin: unquote_literal fix
Marko Kreen [Thu, 16 Dec 2010 13:46:00 +0000 (15:46 +0200)]
qaadmin: unquote_literal fix

14 years agolondiste add: create_flags must be 0 by default
Marko Kreen [Tue, 14 Dec 2010 13:16:48 +0000 (15:16 +0200)]
londiste add: create_flags must be 0 by default

14 years agobump db ver
Marko Kreen [Tue, 14 Dec 2010 12:54:51 +0000 (14:54 +0200)]
bump db ver

14 years agopgq_node: workarounds for bandaid casts
Marko Kreen [Tue, 14 Dec 2010 12:51:21 +0000 (14:51 +0200)]
pgq_node: workarounds for bandaid casts

14 years agopgq: avoid text||int concet, it breaks if db has bandaid casts installed
Marko Kreen [Tue, 14 Dec 2010 12:48:19 +0000 (14:48 +0200)]
pgq: avoid text||int concet, it breaks if db has bandaid casts installed

14 years agolondiste add-table: dont set create_flags prematurely
Marko Kreen [Tue, 14 Dec 2010 12:24:08 +0000 (14:24 +0200)]
londiste add-table: dont set create_flags prematurely

14 years agobump db ver
Marko Kreen [Mon, 13 Dec 2010 08:34:20 +0000 (10:34 +0200)]
bump db ver

14 years agoMerge remote branch 'askotm/master'
Marko Kreen [Mon, 13 Dec 2010 08:02:35 +0000 (10:02 +0200)]
Merge remote branch 'askotm/master'

14 years agofix pg_autovacuum defaults in pgq.tune_storage()
Martin Pihlak [Sun, 12 Dec 2010 13:34:00 +0000 (15:34 +0200)]
fix pg_autovacuum defaults in pgq.tune_storage()

Previous version used zero value for freeze_max_age, which caused
autovacuum to constantly re-vacuum all of the event tables because
of the wraparound threat. Set this and other values to -1 to indicate
system default.

14 years agofixed two typos
Asko Tiidumaa [Thu, 9 Dec 2010 16:27:42 +0000 (16:27 +0000)]
fixed two typos

14 years agowrote handle_fkeys to more gp-friendlier version
Asko Tiidumaa [Thu, 9 Dec 2010 16:25:33 +0000 (16:25 +0000)]
wrote handle_fkeys to more gp-friendlier version

14 years agodbstruct: use new acl parser
Marko Kreen [Wed, 8 Dec 2010 14:39:37 +0000 (16:39 +0200)]
dbstruct: use new acl parser

14 years agoqadmin: remove exception test
Marko Kreen [Wed, 8 Dec 2010 14:37:28 +0000 (16:37 +0200)]
qadmin: remove exception test

14 years agoskytools: parse_acl() func for postgres ACL parsing
Marko Kreen [Wed, 8 Dec 2010 14:33:22 +0000 (16:33 +0200)]
skytools: parse_acl() func for postgres ACL parsing

14 years agolondiste tables: dont show non-local tables
Marko Kreen [Wed, 8 Dec 2010 14:11:50 +0000 (16:11 +0200)]
londiste tables: dont show non-local tables

those can be seen with 'missing'

14 years agoqadmin: catch and show python exceptions
Marko Kreen [Wed, 8 Dec 2010 13:59:04 +0000 (15:59 +0200)]
qadmin: catch and show python exceptions

14 years agolondiste do-sync: actually reset the pgq params
Marko Kreen [Wed, 8 Dec 2010 13:58:36 +0000 (15:58 +0200)]
londiste do-sync: actually reset the pgq params

14 years agotick logic now respects either lag or max tick count, not just both
Asko Tiidumaa [Wed, 8 Dec 2010 13:42:21 +0000 (13:42 +0000)]
tick logic now respects either lag or max tick count, not just both

14 years agopgq_node.version: 3.0.0.4
Marko Kreen [Wed, 8 Dec 2010 12:47:21 +0000 (14:47 +0200)]
pgq_node.version: 3.0.0.4

14 years agoskytools.unquote_ident: dont crash on empty string
Marko Kreen [Wed, 8 Dec 2010 12:45:58 +0000 (14:45 +0200)]
skytools.unquote_ident: dont crash on empty string

14 years agolondiste: force small batches for do-sync
Marko Kreen [Wed, 8 Dec 2010 12:44:57 +0000 (14:44 +0200)]
londiste: force small batches for do-sync

14 years agopgq_node.get_consumer_state: missing returns, update regtest
Marko Kreen [Wed, 8 Dec 2010 12:44:07 +0000 (14:44 +0200)]
pgq_node.get_consumer_state: missing returns, update regtest

14 years agolondiste.playback: unused tablestate method
Marko Kreen [Wed, 8 Dec 2010 12:39:32 +0000 (14:39 +0200)]
londiste.playback: unused tablestate method

14 years agosplit query to avoid gp errors
Asko Tiidumaa [Wed, 8 Dec 2010 11:09:10 +0000 (11:09 +0000)]
split query to avoid gp errors

14 years agolondiste: load bulkloader by default
Marko Kreen [Mon, 6 Dec 2010 12:53:14 +0000 (14:53 +0200)]
londiste: load bulkloader by default

14 years agobulkloader: better error message if wrong event type
Marko Kreen [Mon, 6 Dec 2010 12:47:50 +0000 (14:47 +0200)]
bulkloader: better error message if wrong event type

14 years agoqadmin: sql passthrough, show table/sequence, cleanups
Marko Kreen [Mon, 6 Dec 2010 12:47:00 +0000 (14:47 +0200)]
qadmin: sql passthrough, show table/sequence, cleanups

14 years agodbstruct: slightly more readable SQL output
Marko Kreen [Mon, 6 Dec 2010 12:17:52 +0000 (14:17 +0200)]
dbstruct: slightly more readable SQL output

14 years agoquote_ident: special-case empty string
Marko Kreen [Mon, 6 Dec 2010 12:17:19 +0000 (14:17 +0200)]
quote_ident: special-case empty string

14 years agosql_tokenize: various improvements
Marko Kreen [Mon, 6 Dec 2010 12:14:04 +0000 (14:14 +0200)]
sql_tokenize: various improvements

- drop the use_qident param, wrong place for it
- support multi-symbol operators
- support >2 parts in names
- drop the magic escaping in pynew,pyold.  it does not make sense
- let all unknown symbols count as error

14 years agoqadmin: parser cleanup
Marko Kreen [Fri, 3 Dec 2010 15:27:33 +0000 (17:27 +0200)]
qadmin: parser cleanup

14 years agotodo update
Marko Kreen [Fri, 3 Dec 2010 15:27:12 +0000 (17:27 +0200)]
todo update

14 years agobulkloader doc: show correct londiste switch
Marko Kreen [Fri, 3 Dec 2010 15:26:43 +0000 (17:26 +0200)]
bulkloader doc: show correct londiste switch

14 years agosql_tokenizer: options to returns quoted idents separately
Marko Kreen [Fri, 3 Dec 2010 15:26:10 +0000 (17:26 +0200)]
sql_tokenizer: options to returns quoted idents separately

14 years agotests/handler: bulk handler test
Marko Kreen [Fri, 3 Dec 2010 13:53:51 +0000 (15:53 +0200)]
tests/handler: bulk handler test

14 years agolondiste handler: convert bulkloader to handler
Marko Kreen [Fri, 3 Dec 2010 13:50:03 +0000 (15:50 +0200)]
londiste handler: convert bulkloader to handler

14 years agomagic_insert: allow pre-quoted table name
Marko Kreen [Fri, 3 Dec 2010 13:49:32 +0000 (15:49 +0200)]
magic_insert: allow pre-quoted table name

14 years agorefresh libusual
Marko Kreen [Fri, 3 Dec 2010 12:22:30 +0000 (14:22 +0200)]
refresh libusual

14 years agolondiste playback: fix cursor name
Marko Kreen [Fri, 3 Dec 2010 12:14:08 +0000 (14:14 +0200)]
londiste playback: fix cursor name

14 years agocascadedworker: missing import
Marko Kreen [Fri, 3 Dec 2010 12:13:26 +0000 (14:13 +0200)]
cascadedworker: missing import

14 years agosql/londiste: actually install the .version function
Marko Kreen [Fri, 3 Dec 2010 12:12:59 +0000 (14:12 +0200)]
sql/londiste: actually install the .version function

14 years agolondiste.version(): 3.0.0.6
Marko Kreen [Fri, 3 Dec 2010 09:16:29 +0000 (11:16 +0200)]
londiste.version(): 3.0.0.6

14 years agolondiste copy: at do-sync time disable large batches
Marko Kreen [Fri, 3 Dec 2010 09:15:37 +0000 (11:15 +0200)]
londiste copy: at do-sync time disable large batches

14 years agolondiste.local_add_table: avoid assigning to IN param
Marko Kreen [Fri, 3 Dec 2010 09:02:31 +0000 (11:02 +0200)]
londiste.local_add_table: avoid assigning to IN param

14 years agolondiste.version(): 3.0.0.5
Marko Kreen [Thu, 2 Dec 2010 14:39:23 +0000 (16:39 +0200)]
londiste.version(): 3.0.0.5

14 years agodoc: ignore rsync errors
Marko Kreen [Thu, 2 Dec 2010 14:29:50 +0000 (16:29 +0200)]
doc: ignore rsync errors

14 years agoqadmin: fancier lodiste add_table
Marko Kreen [Thu, 2 Dec 2010 13:46:19 +0000 (15:46 +0200)]
qadmin: fancier lodiste add_table

14 years agolondiste: --no-triggers arg
Marko Kreen [Thu, 2 Dec 2010 13:45:15 +0000 (15:45 +0200)]
londiste: --no-triggers arg

14 years agolondiste.local_add_table: no_triggers and expect_sync magic args
Marko Kreen [Thu, 2 Dec 2010 13:39:50 +0000 (15:39 +0200)]
londiste.local_add_table: no_triggers and expect_sync magic args

14 years agolondiste.version(): 3.0.0.4
Marko Kreen [Wed, 1 Dec 2010 13:25:13 +0000 (15:25 +0200)]
londiste.version(): 3.0.0.4

14 years agotests/londiste: test table-remove on subscribers
Marko Kreen [Wed, 1 Dec 2010 13:21:01 +0000 (15:21 +0200)]
tests/londiste: test table-remove on subscribers

14 years agolondiste: show table counts in 'status'
Marko Kreen [Wed, 1 Dec 2010 13:20:38 +0000 (15:20 +0200)]
londiste: show table counts in 'status'

14 years agosql/londiste: create deny-trigger on leaf
Marko Kreen [Wed, 1 Dec 2010 13:19:34 +0000 (15:19 +0200)]
sql/londiste: create deny-trigger on leaf

14 years agolondiste tests: sync with trigger fmt changes
Marko Kreen [Wed, 1 Dec 2010 13:19:00 +0000 (15:19 +0200)]
londiste tests: sync with trigger fmt changes

14 years agopgq/triggers: 'deny' argument to logutriga/sqltriga
Marko Kreen [Wed, 1 Dec 2010 13:06:12 +0000 (15:06 +0200)]
pgq/triggers: 'deny' argument to logutriga/sqltriga

Needed for deny trigger on leafs.

14 years agolondiste.drop_table_triggers: check whether trigger has args
Marko Kreen [Tue, 30 Nov 2010 13:28:42 +0000 (15:28 +0200)]
londiste.drop_table_triggers: check whether trigger has args

14 years agolondiste.local_add_table: typo fix in 8.3 branch
Marko Kreen [Tue, 30 Nov 2010 13:28:17 +0000 (15:28 +0200)]
londiste.local_add_table: typo fix in 8.3 branch

14 years agorefresh libusual
Marko Kreen [Tue, 30 Nov 2010 13:03:01 +0000 (15:03 +0200)]
refresh libusual

14 years agolondiste.local_add_table: avoid array_length(), its 8.4+
Marko Kreen [Tue, 30 Nov 2010 13:00:02 +0000 (15:00 +0200)]
londiste.local_add_table: avoid array_length(), its 8.4+

14 years agotests/merge: make sync faster
Marko Kreen [Mon, 29 Nov 2010 14:39:49 +0000 (16:39 +0200)]
tests/merge: make sync faster

14 years agolondiste: use pgq.logutriga() as default trigger function
Marko Kreen [Mon, 29 Nov 2010 14:33:21 +0000 (16:33 +0200)]
londiste: use pgq.logutriga() as default trigger function

14 years agotests/merge: wait for sync on combined-root
Marko Kreen [Mon, 29 Nov 2010 14:29:14 +0000 (16:29 +0200)]
tests/merge: wait for sync on combined-root

14 years agolondiste.local_add_table: use --expect-sync on leaf to combined-branch
Marko Kreen [Mon, 29 Nov 2010 14:28:28 +0000 (16:28 +0200)]
londiste.local_add_table: use --expect-sync on leaf to combined-branch

14 years agoset db ver for 3.0.0.3
Marko Kreen [Mon, 29 Nov 2010 13:02:25 +0000 (15:02 +0200)]
set db ver for 3.0.0.3

14 years agoAllow optional use of libevent
Marko Kreen [Mon, 29 Nov 2010 12:44:31 +0000 (14:44 +0200)]
Allow optional use of libevent

14 years agorefresh libevent
Marko Kreen [Mon, 29 Nov 2010 12:44:19 +0000 (14:44 +0200)]
refresh libevent

14 years agoCascadedWorker: merge-leaf must not write to db on branch
Marko Kreen [Mon, 29 Nov 2010 12:43:43 +0000 (14:43 +0200)]
CascadedWorker: merge-leaf must not write to db on branch

14 years agosql/*: Simplify queries to be able to run greenplum as leaf
Asko Tiidumaa [Fri, 26 Nov 2010 12:33:33 +0000 (12:33 +0000)]
sql/*: Simplify queries to be able to run greenplum as leaf

execute:
  Move session_replication_role setting out of db functions.

14 years agopgq.consumer: rename next_batch_custom result fields
Marko Kreen [Fri, 26 Nov 2010 13:05:25 +0000 (15:05 +0200)]
pgq.consumer: rename next_batch_custom result fields

fields names are out-of-sync with get_batch_info()..

14 years agolondiste.local_add_table: skip empty arg array
Marko Kreen [Fri, 26 Nov 2010 11:51:59 +0000 (13:51 +0200)]
londiste.local_add_table: skip empty arg array

14 years agoremove duplicate batch_info setting
Marko Kreen [Fri, 26 Nov 2010 11:32:34 +0000 (13:32 +0200)]
remove duplicate batch_info setting

14 years agolondiste.handlers: separate package for handlers
Marko Kreen [Fri, 19 Nov 2010 13:19:05 +0000 (15:19 +0200)]
londiste.handlers: separate package for handlers

14 years agolondiste: call handler at add-table time
Marko Kreen [Fri, 19 Nov 2010 13:17:51 +0000 (15:17 +0200)]
londiste: call handler at add-table time

14 years agolondiste: some handler fixes
Marko Kreen [Fri, 19 Nov 2010 13:17:27 +0000 (15:17 +0200)]
londiste: some handler fixes

14 years agogitignore: add debian crap
Marko Kreen [Fri, 19 Nov 2010 13:23:27 +0000 (15:23 +0200)]
gitignore: add debian crap

14 years agolondiste: trigger flags to control trigger creation
Marko Kreen [Fri, 12 Nov 2010 14:20:40 +0000 (16:20 +0200)]
londiste: trigger flags to control trigger creation

14 years agomagic_insert: allow shorter rows
Marko Kreen [Fri, 19 Nov 2010 14:13:32 +0000 (16:13 +0200)]
magic_insert: allow shorter rows

14 years agodbstruct: separate handling for column DEFAULTs
Marko Kreen [Fri, 19 Nov 2010 13:14:32 +0000 (15:14 +0200)]
dbstruct: separate handling for column DEFAULTs

14 years agosql/pgq: sanity check against incorrect db state
Marko Kreen [Fri, 12 Nov 2010 14:18:13 +0000 (16:18 +0200)]
sql/pgq: sanity check against incorrect db state

14 years agoqadmin: advanced syntax for londiste cmds, fix more quoting problems
Marko Kreen [Fri, 26 Nov 2010 11:02:10 +0000 (13:02 +0200)]
qadmin: advanced syntax for londiste cmds, fix more quoting problems

14 years agolondiste add-table --create: fix flag handling
Marko Kreen [Fri, 26 Nov 2010 10:04:07 +0000 (12:04 +0200)]
londiste add-table --create: fix flag handling

14 years agofix var name in last commit
Marko Kreen [Fri, 26 Nov 2010 10:01:40 +0000 (12:01 +0200)]
fix var name in last commit

14 years agocascadedconsumer: dont use .finish_remote_batch() in --reset
Marko Kreen [Fri, 26 Nov 2010 09:59:45 +0000 (11:59 +0200)]
cascadedconsumer: dont use .finish_remote_batch() in --reset

14 years agolondiste: play with session_replication_role only on 8.3+
Marko Kreen [Thu, 25 Nov 2010 15:47:10 +0000 (17:47 +0200)]
londiste: play with session_replication_role only on 8.3+

14 years agolondiste worker --reset fixes
Marko Kreen [Thu, 25 Nov 2010 15:44:54 +0000 (17:44 +0200)]
londiste worker --reset fixes

14 years agopgq.version: 3.0.0.2
Marko Kreen [Thu, 25 Nov 2010 13:07:48 +0000 (15:07 +0200)]
pgq.version: 3.0.0.2

14 years agotests/merge: some improvements
Marko Kreen [Thu, 25 Nov 2010 13:07:05 +0000 (15:07 +0200)]
tests/merge: some improvements

14 years agopgq.drop_queue: make force drop old consumers
Marko Kreen [Thu, 25 Nov 2010 13:06:28 +0000 (15:06 +0200)]
pgq.drop_queue: make force drop old consumers

14 years agoqadmin; better quoting support
Marko Kreen [Thu, 25 Nov 2010 12:19:18 +0000 (14:19 +0200)]
qadmin; better quoting support

14 years agounquote_ident: lowercase the non-quoted value
Marko Kreen [Thu, 25 Nov 2010 12:10:13 +0000 (14:10 +0200)]
unquote_ident: lowercase the non-quoted value

Otherwise the value will later be quoted incorrectly.

14 years agoskytools.sql_tokenizer: optionally return token position
Marko Kreen [Thu, 25 Nov 2010 12:04:43 +0000 (14:04 +0200)]
skytools.sql_tokenizer: optionally return token position

14 years agoskytools.unquote_(fq)ident: tests, relax fq
Marko Kreen [Thu, 25 Nov 2010 12:03:52 +0000 (14:03 +0200)]
skytools.unquote_(fq)ident: tests, relax fq

unquote_fqident does not require both parts anymore

14 years agolondiste setup: only set replica role for EXECUTE transaction
Marko Kreen [Thu, 25 Nov 2010 11:58:06 +0000 (13:58 +0200)]
londiste setup: only set replica role for EXECUTE transaction

otherwise cleanup triggers wont launch

14 years agoqadmin: londiste commands
Marko Kreen [Thu, 25 Nov 2010 09:53:17 +0000 (11:53 +0200)]
qadmin: londiste commands

add_table, remove_table, tables,
add_seq, remove_seq, seqs,
missing

14 years agoskytools.sql_tokenizer: support fully-qualified names
Marko Kreen [Thu, 25 Nov 2010 07:24:25 +0000 (09:24 +0200)]
skytools.sql_tokenizer: support fully-qualified names

It's easier to merge name parts here than to do it in upper layers.

The merging does not support some crazier parts of sql syntax
like comments between name parts, so it defaults to off.

14 years agopgq_node.version() - 3.0.0.2
Marko Kreen [Wed, 24 Nov 2010 20:29:25 +0000 (22:29 +0200)]
pgq_node.version() - 3.0.0.2