doc update

diff --git a/doc/Makefile b/doc/Makefile
index 0fa9930b712218f37d6e2f57203fd5ad20f4c52c..a16b89e241c4938179bfbfa1b260cf4bc031f091 100644 (file)
--- a/doc/Makefile
+++ b/doc/Makefile
@@ -11,7 +11,7 @@ EPYARGS = --no-private --url="http://pgfoundry.org/projects/skytools/" \
 
 HTMLS = londiste.cmdline.html londiste.config.html README.html INSTALL.html \
        londiste.ref.html TODO.html pgq-sql.html pgq-admin.html pgq-nodupes.html \
-       $(SCRIPT_HTMLS)
+       $(SCRIPT_HTMLS) skytools3.html
 
 SCRIPT_TXTS = walmgr.txt cube_dispatcher.txt table_dispatcher.txt \
              queue_mover.txt queue_splitter.txt bulk_loader.txt \

diff --git a/doc/TODO.txt b/doc/TODO.txt
index 601fb7bbcb652949404450f68b1654eb4404425c..0ddd28750f63f9ef5abcc73170ef41228f4c4da1 100644 (file)
--- a/doc/TODO.txt
+++ b/doc/TODO.txt
@@ -22,10 +22,12 @@
      - resume
      - node-status
      - set-status
+     - rename-node
    - standard msg/error handling for all sql functions
    - compare/repair
    - check if table is in other sets? [NAK]
    - setconsumer/pgq - insert tick with original date (+evid?)
+   - dont trust node name in member_info, set_info is authoritative
 
  * drop support for 8.1 ??
 

diff --git a/doc/skytools3.txt b/doc/skytools3.txt
index 6eacdf5efd6d730b5fb65b956f87f7d5d5402fcb..13b92c091e80b5d662cf873f77f815c9887c0b93 100644 (file)
--- a/doc/skytools3.txt
+++ b/doc/skytools3.txt
@@ -1,19 +1,53 @@
 
-Keep old design from Skytools 2.x:
-----------------------------------
+Skytools 3 - cascaded replication
+=================================
 
-* worker process connects to only 2 databases
-* worker process pulls data, no pushing with LISTEN/NOTIFY
-* relaxed attitude about tables
-  - can be added/removed any time.
-  - inital sync happens table-by-table, no attempt is made to keep
-    consistend picture between tables when syncing.
+Keep old design from Skytools 2
+-------------------------------
 
-New features in 3.0:
---------------------
+* Worker process connects to only 2 databases, there is no
+  everybody-to-everybody communication going on.
+* Worker process only pulls data from queue.
+  - No pushing with LISTEN/NOTIFY
+  - Administrative work happens in separate process
+  - Can go down anytime, without affecting anything else
+* Relaxed attitude about tables
+  - Tables can be added/removed any time.
+  - Inital sync happens table-by-table, no attempt is made to keep
+    consistent picture between tables when syncing.
 
-* Queue content are kept in sync between all nodes in node-set.
-* parallel copy - during inital sync several tables can be
+New features in Skytools 3
+--------------------------
+
+* Cascading is implemented as generic layer on top of PgQ - *pgq_set*.
+  - Its goal is to keep identical copy of queue contents in several nodes.
+  - Not replication-specific - can be used for any queue.
+
+* Parallel copy - during inital sync several tables can be
   copied at the same time.
+  - In 2.1 the copy already happened in separate process,
+    making it parallel was just a matter of tuning launching/syncing logic.
+
+* Advanced admin operations:
+  - switchover
+  - failover
+  - change provider for node
+  - pause / resume node
+  - rename node
+
+Open Questions
+--------------
+
+* terminology
+  - 'set' vs 'cluster' vs ???
+  - (root / branch / leaf) vs. (master / slave) vs. (provider / subscriber)
+  - add <table> vs. attach <table>
+
+* own trigger handling
+  - fullblown custom
+  - minimal custom + wait for 8.3
 
+* compatibility with v2.1.x (python scripts + modules):
+  - bundle old modules and scripts with different name
+  - rename new modules and scripts