skytools.git
16 years agoupdate TODO, fix londiste doc
Marko Kreen [Mon, 1 Jun 2009 13:01:37 +0000 (16:01 +0300)]
update TODO, fix londiste doc

16 years agolondiste: local_add_seq() - log seq name
Marko Kreen [Mon, 1 Jun 2009 13:01:01 +0000 (16:01 +0300)]
londiste: local_add_seq() - log seq name

16 years agolondiste: globbing, --all
Marko Kreen [Mon, 1 Jun 2009 12:51:54 +0000 (15:51 +0300)]
londiste: globbing, --all

16 years agolondiste: fix resync / missing commands
Marko Kreen [Mon, 1 Jun 2009 09:30:33 +0000 (12:30 +0300)]
londiste: fix resync / missing commands

16 years agopython scripts: Move template config to docstring.
Asko Oja [Mon, 4 May 2009 14:02:39 +0000 (14:02 +0000)]
python scripts: Move template config to docstring.

* Template config is now in docstring to make to more
  easy to keep up-to-date.

* dbscript: --ini option to show template config.

16 years agopgqd: use strlist
Marko Kreen [Mon, 1 Jun 2009 06:25:56 +0000 (09:25 +0300)]
pgqd: use strlist

16 years agopgqd: support database_list
Marko Kreen [Mon, 1 Jun 2009 06:14:41 +0000 (09:14 +0300)]
pgqd: support database_list

16 years agoupgrade libusual
Marko Kreen [Mon, 1 Jun 2009 07:12:49 +0000 (10:12 +0300)]
upgrade libusual

16 years agocascade: more takeover work
Marko Kreen [Fri, 29 May 2009 11:46:01 +0000 (14:46 +0300)]
cascade: more takeover work

16 years agodocheck: allow files from cmdline
Marko Kreen [Wed, 13 May 2009 14:23:39 +0000 (17:23 +0300)]
docheck: allow files from cmdline

16 years agodbscript: better desc for -q/-v
Marko Kreen [Wed, 13 May 2009 14:23:01 +0000 (17:23 +0300)]
dbscript: better desc for -q/-v

16 years agocascade status: work on broken tree, show node type
Marko Kreen [Wed, 13 May 2009 11:15:13 +0000 (14:15 +0300)]
cascade status: work on broken tree, show node type

- Show node type after name

- Allow several roots, take nodes with missing parents as roots

It still cannot work on cycles.  But as the change-provider code
should avoid making those, maybe it's not worth worrying about.

16 years agotests: move common test code to under tests/*.sh
Marko Kreen [Wed, 13 May 2009 10:22:49 +0000 (13:22 +0300)]
tests: move common test code to under tests/*.sh

16 years agocascade: takeover
Marko Kreen [Fri, 24 Apr 2009 14:33:39 +0000 (17:33 +0300)]
cascade: takeover

Instead of switchover/failover commands have 'takeover' command,
to be launched from new node.

16 years agodoc updates
Marko Kreen [Mon, 4 May 2009 12:35:02 +0000 (15:35 +0300)]
doc updates

16 years agoMerge commit 'askotm/master'
Marko Kreen [Wed, 6 May 2009 10:36:20 +0000 (13:36 +0300)]
Merge commit 'askotm/master'

16 years agoDBScript: bad argument reference in run_safely
Marko Kreen [Wed, 6 May 2009 10:28:12 +0000 (13:28 +0300)]
DBScript: bad argument reference in run_safely

16 years agoCascadeAdmin: add missing import
Marko Kreen [Wed, 6 May 2009 10:27:35 +0000 (13:27 +0300)]
CascadeAdmin: add missing import

16 years agoskytools.Config: getdict() method
Marko Kreen [Wed, 6 May 2009 10:26:50 +0000 (13:26 +0300)]
skytools.Config: getdict() method

16 years agosql/*: update regtests with new messages
Marko Kreen [Mon, 4 May 2009 11:15:46 +0000 (14:15 +0300)]
sql/*: update regtests with new messages

16 years agofixed error message
Asko Tiidumaa [Wed, 6 May 2009 08:01:47 +0000 (08:01 +0000)]
fixed error message

16 years agofixed error message
Asko Tiidumaa [Wed, 6 May 2009 08:01:08 +0000 (08:01 +0000)]
fixed error message

16 years agofixed two error messages that were copied from log(u)triga
Asko Tiidumaa [Wed, 6 May 2009 08:00:13 +0000 (08:00 +0000)]
fixed two error messages that were copied from log(u)triga

16 years agoAdd querybuilder into sql tools
Asko Oja [Mon, 4 May 2009 12:22:25 +0000 (12:22 +0000)]
Add querybuilder into sql tools

Querybulder is used to manage parametrisized queries both in
plpython/dbservice stored procedures and python scripts.

16 years agoMerge commit 'martin/master'
Marko Kreen [Mon, 4 May 2009 11:06:47 +0000 (14:06 +0300)]
Merge commit 'martin/master'

16 years agopgq_node, londiste: improve log messages from db
Marko Kreen [Mon, 4 May 2009 10:47:59 +0000 (13:47 +0300)]
pgq_node, londiste: improve log messages from db

londiste.global_add_table()
londiste.global_remove_table()
pgq_node.set_global_watermark()

 - Replace 'Ok' with actual sentence

londiste.root_check_seqs():
 - improved message, downgrade to debug level

pgq_node.set_subscriber_watermark():
 - improve message
 - check tick value sanity

16 years agopython/pgq/cascade: proper process_root_node() calling
Marko Kreen [Mon, 4 May 2009 10:35:47 +0000 (13:35 +0300)]
python/pgq/cascade: proper process_root_node() calling

CascadedWorker did not call parent class method.

Also remove pointless logging from CascadedConsumer.

16 years agopgq.register_consumer_at(): update last seen timestamp on position move
Marko Kreen [Mon, 4 May 2009 10:31:02 +0000 (13:31 +0300)]
pgq.register_consumer_at(): update last seen timestamp on position move

16 years agoINSTALL: automake seems needed for aclocal
Marko Kreen [Mon, 4 May 2009 08:39:29 +0000 (11:39 +0300)]
INSTALL: automake seems needed for aclocal

16 years agotests/londiste: make logging verbose
Marko Kreen [Mon, 4 May 2009 07:48:56 +0000 (10:48 +0300)]
tests/londiste: make logging verbose

16 years agocascade: make wm publish periods configurable
Marko Kreen [Mon, 4 May 2009 07:37:00 +0000 (10:37 +0300)]
cascade: make wm publish periods configurable

This is mostly meant for testing.  The options may be removed or
renamed in the future.

Also fix bug in local wm publish function

16 years agocascade: fix watermark publishing
Marko Kreen [Sat, 2 May 2009 11:47:39 +0000 (14:47 +0300)]
cascade: fix watermark publishing

* CascadedWorker:
  - fix time comparision in local wm handling
  - publish wm also for leaf nodes
  - fields node_type, node_name, local_watermark were missing in WorkerState
* pgq_node.get_node_info: on leaf node set local watermark to last consumer tick
* pgq_node.set_global_watermark: don't depend on exact code from pgq_node.get_node_info()

Trying to special-case leaf nodes does not seem to be good idea, because
then provider nodes also need to know subscriber node type.  So stop doing it.

16 years agoadded a small useful example consumer
Asko Tiidumaa [Thu, 30 Apr 2009 11:19:19 +0000 (11:19 +0000)]
added a small useful example consumer

16 years agoMerge branch 'master' of git://github.com/markokr/skytools-dev
Asko Tiidumaa [Thu, 30 Apr 2009 11:17:37 +0000 (11:17 +0000)]
Merge branch 'master' of git://github.com/markokr/skytools-dev

16 years agoQueue connection improvements.
Martin Pihlak [Thu, 30 Apr 2009 10:50:49 +0000 (13:50 +0300)]
Queue connection improvements.

- Ability to disconnect from queue by specifying empty qname.
- Verify that the queue exists before connecting.

16 years agosql/ticker: make routine database detection work.
Marko Kreen [Thu, 30 Apr 2009 10:06:54 +0000 (13:06 +0300)]
sql/ticker: make routine database detection work.

Previously databases were only detected at startup.

16 years agoFinalize newadm to qadmin rename.
Martin Pihlak [Thu, 30 Apr 2009 09:53:56 +0000 (12:53 +0300)]
Finalize newadm to qadmin rename.

16 years agoCreate documentation directory before attempting to install.
Martin Pihlak [Thu, 30 Apr 2009 09:32:49 +0000 (12:32 +0300)]
Create documentation directory before attempting to install.

16 years ago"newadm" renamed to "qadmin"
Martin Pihlak [Thu, 30 Apr 2009 09:10:22 +0000 (12:10 +0300)]
"newadm" renamed to "qadmin"
Documentation and install scripts modified accordingly.

16 years agolondiste: cmd_execute() did not check ret_code of londiste.execute_start()
Marko Kreen [Thu, 30 Apr 2009 09:06:11 +0000 (12:06 +0300)]
londiste: cmd_execute() did not check ret_code of londiste.execute_start()

Also unify checking in both places - dont depend on exactly
'200' code from there.

16 years agoMerge branch 'master' of git://github.com/markokr/skytools-dev
Martin Pihlak [Thu, 30 Apr 2009 08:42:31 +0000 (11:42 +0300)]
Merge branch 'master' of git://github.com/markokr/skytools-dev

16 years agolondiste copy: don't give -q unconditionally to copy process.
Marko Kreen [Wed, 29 Apr 2009 15:20:34 +0000 (18:20 +0300)]
londiste copy: don't give -q unconditionally to copy process.

Seems the original logic assument pgbouncer-like -q,
which only disabled stdout.  But DBScript sets
global log level with it.

16 years agoMerge branch 'master' of git://github.com/markokr/skytools-dev
Martin Pihlak [Wed, 29 Apr 2009 12:14:02 +0000 (15:14 +0300)]
Merge branch 'master' of git://github.com/markokr/skytools-dev

Conflicts:

python/pgq/cascade/admin.py

16 years agoMerge commit 'origin/master'
Asko Tiidumaa [Wed, 29 Apr 2009 12:12:11 +0000 (12:12 +0000)]
Merge commit 'origin/master'

16 years agoMerge commit 'priit/master'
Marko Kreen [Wed, 29 Apr 2009 08:50:58 +0000 (11:50 +0300)]
Merge commit 'priit/master'

16 years agolondiste: unregister fix in playback.py
Priit Kustala [Tue, 28 Apr 2009 13:35:36 +0000 (13:35 +0000)]
londiste: unregister fix in playback.py

16 years agosql/ticker: ifdef SIGPIPE, SIGHUP for win32
Marko Kreen [Tue, 28 Apr 2009 19:53:44 +0000 (22:53 +0300)]
sql/ticker: ifdef SIGPIPE, SIGHUP for win32

16 years agocquoting: compat macros for MSC, minor cleanups
Marko Kreen [Tue, 28 Apr 2009 19:48:08 +0000 (22:48 +0300)]
cquoting: compat macros for MSC, minor cleanups

- MSC: macros for 'inline', 'strcasecmp'
- Use stdbool.h instead custom enum for 'bool'.

16 years agodevnotes: better formatting
Marko Kreen [Tue, 28 Apr 2009 19:44:33 +0000 (22:44 +0300)]
devnotes: better formatting

16 years agotag the tree as 3.0git
Marko Kreen [Tue, 28 Apr 2009 19:43:47 +0000 (22:43 +0300)]
tag the tree as 3.0git

16 years agosql/: Use more portable sed expression to get list of files.
Marko Kreen [Tue, 28 Apr 2009 19:42:26 +0000 (22:42 +0300)]
sql/: Use more portable sed expression to get list of files.

Old expression was not sed 3.x compatible, used in mingw.

16 years agolondiste add-table: if schema is missing, create
Marko Kreen [Tue, 28 Apr 2009 11:45:46 +0000 (14:45 +0300)]
londiste add-table: if schema is missing, create

16 years agodoc/devnotes: describe git usage more
Marko Kreen [Tue, 28 Apr 2009 11:44:55 +0000 (14:44 +0300)]
doc/devnotes: describe git usage more

16 years agodocs: nicer code blocks
Marko Kreen [Tue, 28 Apr 2009 11:44:27 +0000 (14:44 +0300)]
docs: nicer code blocks

16 years agoskytools.Config: use config file name as default job_name
Marko Kreen [Mon, 27 Apr 2009 19:49:46 +0000 (22:49 +0300)]
skytools.Config: use config file name as default job_name

16 years agoDBScript: avoid double config reload
Marko Kreen [Tue, 28 Apr 2009 11:51:10 +0000 (14:51 +0300)]
DBScript: avoid double config reload

also remove pointless job_name default, as it is already
handled in Config class.

16 years agoscripting: unify exception handling, show db name
Marko Kreen [Fri, 24 Apr 2009 14:32:28 +0000 (17:32 +0300)]
scripting: unify exception handling, show db name

Have single place where exceptions are handled.

psycopg attaches .cursor to each exception,
use that to show connection name in error messages.

16 years agoDecrease log level in pgq_node.get_node_info.sql and pgq_node.get_consumer_state.sql
Priit Kustala [Tue, 28 Apr 2009 10:25:38 +0000 (10:25 +0000)]
Decrease log level in pgq_node.get_node_info.sql and pgq_node.get_consumer_state.sql

16 years agofix in admin.py in get_node_subscriber_list ... remove local_watermark column from...
Priit Kustala [Mon, 27 Apr 2009 13:44:58 +0000 (13:44 +0000)]
fix in admin.py in get_node_subscriber_list ... remove local_watermark column from select

16 years agohello world
AskoTiidumaa [Mon, 27 Apr 2009 10:51:19 +0000 (10:51 +0000)]
hello world

16 years ago..
AskoTiidumaa [Mon, 27 Apr 2009 10:41:34 +0000 (10:41 +0000)]
..

16 years agoSupport for drop-node.
Martin Pihlak [Fri, 24 Apr 2009 07:51:18 +0000 (10:51 +0300)]
Support for drop-node.

16 years agopgq: some cleanups
Marko Kreen [Wed, 22 Apr 2009 14:13:31 +0000 (17:13 +0300)]
pgq: some cleanups

also remove silly co_name default.

16 years agosql/pgq: pgq.batch_retry()
Marko Kreen [Tue, 21 Apr 2009 19:57:03 +0000 (22:57 +0300)]
sql/pgq: pgq.batch_retry()

For use by pgq_coop.

16 years agopgq: Drop failed event handling.
Marko Kreen [Tue, 21 Apr 2009 10:08:08 +0000 (13:08 +0300)]
pgq: Drop failed event handling.

Badly designed and unused concept.  If such thing is needed
it's much better to handle them via actual queue.

16 years agoSimple "show node" support.
Martin Pihlak [Tue, 21 Apr 2009 09:52:00 +0000 (12:52 +0300)]
Simple "show node" support.

16 years agoskytools3.txt: link to londiste demo
Marko Kreen [Mon, 20 Apr 2009 09:09:45 +0000 (12:09 +0300)]
skytools3.txt: link to londiste demo

16 years agoticker: fix cfparser conversion
Marko Kreen [Mon, 20 Apr 2009 09:06:42 +0000 (12:06 +0300)]
ticker: fix cfparser conversion

Agile copy-paste hurts.

Also mention heap-impl and base for libusual,
to get proper header dependencies.

16 years agoUpgrade to newer libusual
Marko Kreen [Mon, 20 Apr 2009 07:16:44 +0000 (10:16 +0300)]
Upgrade to newer libusual

- Use autoconf macros from usual.m4
- generate lib/usual/config.h
- ticker: use newer cfparser API
- ticker: use compat module

16 years agoticker: remove obsolete Assert()
Marko Kreen [Mon, 20 Apr 2009 07:10:40 +0000 (10:10 +0300)]
ticker: remove obsolete Assert()

16 years agoRefresh libusual
Marko Kreen [Wed, 15 Apr 2009 17:36:01 +0000 (20:36 +0300)]
Refresh libusual

16 years agofix query output formatting.
Martin Pihlak [Wed, 15 Apr 2009 13:58:53 +0000 (16:58 +0300)]
fix query output formatting.

16 years agoAdd ; to commands strings specified via -c
Martin Pihlak [Wed, 15 Apr 2009 13:29:43 +0000 (16:29 +0300)]
Add ; to commands strings specified via -c

16 years agoAllow queue name to be specified for register/unregister consumer.
Martin Pihlak [Wed, 15 Apr 2009 12:53:12 +0000 (15:53 +0300)]
Allow queue name to be specified for register/unregister consumer.

16 years agoAllow quoted queue and consumer names.
Martin Pihlak [Tue, 14 Apr 2009 14:00:02 +0000 (17:00 +0300)]
Allow quoted queue and consumer names.

16 years agoMerge branch 'master' of git://github.com/markokr/skytools-dev
Martin Pihlak [Tue, 14 Apr 2009 10:19:44 +0000 (13:19 +0300)]
Merge branch 'master' of git://github.com/markokr/skytools-dev

16 years agotests/londiste: reintroduce lost topo tests
Marko Kreen [Mon, 13 Apr 2009 20:19:51 +0000 (23:19 +0300)]
tests/londiste: reintroduce lost topo tests

16 years agodoc: move html cleaning to realclean skytools_3_0_a1
Marko Kreen [Mon, 13 Apr 2009 18:17:29 +0000 (21:17 +0300)]
doc: move html cleaning to realclean

16 years agotgz generation fixes
Marko Kreen [Mon, 13 Apr 2009 18:02:48 +0000 (21:02 +0300)]
tgz generation fixes

- include html files
- include scripts/*.ini
- include misc/*
- include *.ini under sql/

16 years agoMerge branch 'master' of git@github.com:markokr/skytools-dev
Marko Kreen [Mon, 13 Apr 2009 17:29:37 +0000 (20:29 +0300)]
Merge branch 'master' of git@github.com:markokr/skytools-dev

16 years agolast doc update
Marko Kreen [Mon, 13 Apr 2009 17:28:50 +0000 (20:28 +0300)]
last doc update

16 years agoMerge branch 'master' of git@github.com:markokr/skytools-dev
Marko Kreen [Mon, 13 Apr 2009 17:27:00 +0000 (20:27 +0300)]
Merge branch 'master' of git@github.com:markokr/skytools-dev

16 years agopgq/triggers: remove double include
Marko Kreen [Mon, 13 Apr 2009 17:22:00 +0000 (20:22 +0300)]
pgq/triggers: remove double include

Both I and Martin added the builtins.h.

16 years agotests/londiste: new tests
Marko Kreen [Mon, 13 Apr 2009 17:12:54 +0000 (20:12 +0300)]
tests/londiste: new tests

16 years agosql/ticker: accept subsecond timeouts
Marko Kreen [Mon, 13 Apr 2009 14:26:29 +0000 (17:26 +0300)]
sql/ticker: accept subsecond timeouts

also update it's todo

16 years agoMerge branch 'master' of git://github.com/markokr/skytools-dev
Marko Kreen [Mon, 13 Apr 2009 14:17:40 +0000 (17:17 +0300)]
Merge branch 'master' of git://github.com/markokr/skytools-dev

16 years agotag tree as 3.0a1
Marko Kreen [Mon, 13 Apr 2009 14:17:12 +0000 (17:17 +0300)]
tag tree as 3.0a1

16 years agosql/ticker: make timeouts work
Marko Kreen [Mon, 13 Apr 2009 14:15:51 +0000 (17:15 +0300)]
sql/ticker: make timeouts work

16 years agosource.cfg: include libusual in tgz
Marko Kreen [Mon, 13 Apr 2009 14:15:07 +0000 (17:15 +0300)]
source.cfg: include libusual in tgz

16 years agoconfigure.ac: Detect working gcc warning flags
Marko Kreen [Mon, 13 Apr 2009 14:14:35 +0000 (17:14 +0300)]
configure.ac: Detect working gcc warning flags

16 years agodocs: more updates
Marko Kreen [Thu, 9 Apr 2009 13:58:07 +0000 (16:58 +0300)]
docs: more updates

New devnotes.txt to help developers.

16 years agoImport fixes from 2.1-stable.
Marko Kreen [Thu, 9 Apr 2009 09:25:09 +0000 (12:25 +0300)]
Import fixes from 2.1-stable.

- make skytools pkg depend on rsync
- missing headers in Postgres modules
- quoting for numeric ident

16 years agotop Makefile: instead of explicit targets, use subdir targets
Marko Kreen [Mon, 13 Apr 2009 13:05:58 +0000 (16:05 +0300)]
top Makefile: instead of explicit targets, use subdir targets

16 years agodoc: update INSTALL on how to build from git
Marko Kreen [Mon, 13 Apr 2009 13:05:18 +0000 (16:05 +0300)]
doc: update INSTALL on how to build from git

16 years agosql/ticker: make pgqd compile & install without PGXS
Marko Kreen [Mon, 13 Apr 2009 12:42:38 +0000 (15:42 +0300)]
sql/ticker: make pgqd compile & install without PGXS

PGXS used Postgres prefix, but we want executables to use
skytools prefix.

This needed also C compilation support from autoconf.
Lift it from pgbouncer.

16 years agosql/ticker: Convert pgqd to libusual.
Marko Kreen [Mon, 13 Apr 2009 11:35:34 +0000 (14:35 +0300)]
sql/ticker: Convert pgqd to libusual.

Thus it aquires:
- daemonization
- logging
- config file
- event loop without full libevent

16 years agoadded missing header file "access/xact.h"
Martin Pihlak [Mon, 13 Apr 2009 11:25:05 +0000 (14:25 +0300)]
added missing header file "access/xact.h"

16 years agolib: Add libusual as subproject
Marko Kreen [Mon, 13 Apr 2009 10:05:31 +0000 (13:05 +0300)]
lib: Add libusual as subproject

16 years agoMerge commit 'martin/master'
Marko Kreen [Mon, 13 Apr 2009 10:03:08 +0000 (13:03 +0300)]
Merge commit 'martin/master'

16 years agotests: move londiste test from cascade/ to londiste/
Marko Kreen [Wed, 8 Apr 2009 16:40:44 +0000 (19:40 +0300)]
tests: move londiste test from cascade/ to londiste/