Bo Peng [Mon, 17 Aug 2020 05:49:12 +0000 (14:49 +0900)]
Doc: Change sample scripts and yum repository urls.
Bo Peng [Mon, 17 Aug 2020 04:46:36 +0000 (13:46 +0900)]
Use pg_config to get PostgreSQL lib path and add this path to LD_LIBRARY_PATH environment paramater in regress.sh.
Tatsuo Ishii [Mon, 17 Aug 2020 00:09:13 +0000 (09:09 +0900)]
Doc: add description to "show pool_backend_stats".
Mention that failed commands are also counted.
Tatsuo Ishii [Sun, 16 Aug 2020 22:47:11 +0000 (07:47 +0900)]
Fix error count stats.
Unnecessary error count stats collecting route was left in commit:
f04ef82be528c5e5e706f9e2e7231db369427eee.
Tatsuo Ishii [Sat, 15 Aug 2020 09:01:38 +0000 (18:01 +0900)]
Fix pgpool_setup so that native replication mode and SI mode works with PostgreSQL 12.
In these mode, it created recovery.conf which is not allowed in
PostgreSQL 12.
Tatsuo Ishii [Sat, 15 Aug 2020 03:37:39 +0000 (12:37 +0900)]
Fix oversight in connection_life_time fix.
Commit
39eb90fe199d03253b99bec38d85c40f66aa55ab did not consider the case
when node is down connection struct is NULL. As a result, segfault occurred.
Tatsuo Ishii [Fri, 14 Aug 2020 12:07:05 +0000 (21:07 +0900)]
Doc: add Japanese manual for "show pool_backend_stats".
Tatsuo Ishii [Fri, 14 Aug 2020 09:00:55 +0000 (18:00 +0900)]
Fix error stats.
Error stats was only collected from Master node. It is possible that
error occurred in multiple backends (for example, in native
replication mode, INSERT is accepted by the parser but target table
does not exist). Change the place where to collect error stats from
SimpleForwardToFrontend() to read_kind_from_backend().
Tatsuo Ishii [Fri, 14 Aug 2020 06:42:58 +0000 (15:42 +0900)]
Add error stats from backend to "show pool_backend_stats" command.
Tatsuo Ishii [Fri, 14 Aug 2020 05:15:22 +0000 (14:15 +0900)]
Check panic or fatal error by using 'V' token if available.
To identify panic or fatal message in is_panic_or_fatal_error(), 'S'
token was used. However 'S' could be localized and simply comparing
"PANIC" or "FATAL" with the message may not work. Instead 'V' should be
used if possible because it's not localized.
Tatsuo Ishii [Fri, 14 Aug 2020 04:53:57 +0000 (13:53 +0900)]
Add new show command "show pool_backend_stats".
The new command shows the number SQL commands executed since Pgpool-II
started.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-July/003754.html
Bo Peng [Fri, 14 Aug 2020 02:18:25 +0000 (11:18 +0900)]
Fix 031.connection_life_time regression test failure.
Tatsuo Ishii [Fri, 14 Aug 2020 01:45:54 +0000 (10:45 +0900)]
Add statistics for INSERT/UPDATE/DELETE etc. SQL commands.
We already allow to collect stats for SELECT, but it would be useful
to collect stats for other types of SQL commands. APIs to fetch stats
data is also added. No UI is added yet. Currently, any SQL commands
(besides SELECT/INSERT/UPDATE/DELETE) other than:
case(T_CheckPointStmt):
case(T_DeallocateStmt):
case(T_DiscardStmt):
case(T_ExecuteStmt):
case(T_ExplainStmt):
case(T_ListenStmt):
case(T_LoadStmt):
case(T_LockStmt):
case(T_NotifyStmt):
case(T_PrepareStmt):
case(T_TransactionStmt):
case(T_UnlistenStmt):
case(T_VacuumStmt):
case(T_VariableSetStmt):
case(T_VariableShowStmt):
are counted as DDL commands.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-July/003754.html
Tatsuo Ishii [Fri, 14 Aug 2020 01:29:21 +0000 (10:29 +0900)]
Refactor "show pool_nodes" command.
Now number of codes in nodes_reporting() is dramatically reduced by
introducing send_row_description_and_data_rows(), a workhorse to
produce reports. Eventually other reporting functions will be
refactored as well.
Bo Peng [Thu, 13 Aug 2020 06:35:34 +0000 (15:35 +0900)]
Before running regression test clean the remaining processes and sockets.
Tatsuo Ishii [Wed, 12 Aug 2020 11:17:44 +0000 (20:17 +0900)]
Doc: mention that ssl_ciphers only affects to TLS 1.2 and lower.
The changes are imported from PostgreSQL manual.
Bo Peng [Wed, 12 Aug 2020 07:45:27 +0000 (16:45 +0900)]
Remove some unnecessary settings from sample config files.
Bo Peng [Wed, 12 Aug 2020 07:07:04 +0000 (16:07 +0900)]
At pgpool startup, if pid file exists, truncate pid file to zero length before writing.
Patch is created by maliangzhu.
Bo Peng [Tue, 11 Aug 2020 14:58:51 +0000 (23:58 +0900)]
Doc: fix failback_command documetation error.
Bo Peng [Tue, 11 Aug 2020 14:18:39 +0000 (23:18 +0900)]
Doc: add sample script links in configuration example.
Bo Peng [Tue, 11 Aug 2020 09:17:42 +0000 (18:17 +0900)]
Make RPMs to include pcp.conf and pool_hba.conf sample files.
Bo Peng [Tue, 11 Aug 2020 08:34:49 +0000 (17:34 +0900)]
Make RPMs to include new sample files.
Tatsuo Ishii [Tue, 11 Aug 2020 01:26:43 +0000 (10:26 +0900)]
Fix connection_life_time does not work.
If master node is not 0 (this could happen in the case when primary
node is not 0 in streaming replication mode for example), pgpool
failed to find connection_life_time timer because the timer was set
only in virtual_master_node, which could be changed after session ends
since query context is not there any more.
To fix this, connection_life_time timer is set to all valid nodes.
Regression test is also added.
Discussion: https://www.pgpool.net/pipermail/pgpool-general/2020-August/007242.html
Bo Peng [Mon, 10 Aug 2020 15:57:46 +0000 (00:57 +0900)]
New feature: Simplify Watchdog related configuration parameters.
This commit simplifies watchdog related configuration parameters
by using a common config file for each pgpool node and a pgpool_node_id file.
Previously we need to configure the following parameters on each pgpool node,
because of the difference between local and remote pgpool nodes settings.
- wd_hostname
- wd_port
- wd_heartbeat_port
- heartbeat_device
- heartbeat_destination0
- heartbeat_destination_port0
- heartbeat_destination1
- heartbeat_destination_port1
- other_pgpool_hostname0
- other_pgpool_port0
- other_wd_port0
- other_pgpool_hostname1
- other_pgpool_port1
- other_wd_port1
This commit simplifies watchdog related configuration by using a common config file
for each pgpool node and a pgpool_node_id file, and users just copy the configuration file
to each pgpool node without editing.
The parameters above are changed to (for 3 pgpool nodes):
(For example)
==============================
hostname0 = 'server1'
wd_port0 = 9000
pgpool_port0 = 9999
hostname1 = 'server2'
wd_port1 = 9000
pgpool_port1 = 9999
hostname2 = 'server3'
wd_port2 = 9000
pgpool_port2 = 9999
heartbeat_hostname0 = 'server1'
heartbeat_port0 = 9694
heartbeat_device0 = ''
heartbeat_hostname1 = 'server2'
heartbeat_port1 = 9694
heartbeat_device1 = ''
heartbeat_hostname2 = 'server3'
heartbeat_port2 = 9694
heartbeat_device2 = ''
==============================
You can specify multiple configurations in "heartbeat_hostname" and "heartbeat_device" by separating
them using semicolon (;).
And user needs to specify local pgpool node id in a pgpool_node_id file on each pgpool node.
pgpool_node_id file should be created in the direcroty of pgpool.conf.
In addition, this commit increases "WD_MESSAGE_DATA_VERSION_MINOR" to "2".
Tatsuo Ishii [Fri, 7 Aug 2020 00:21:34 +0000 (09:21 +0900)]
Fix oversight in previous commit for pgpool_setup.
Commit:
https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=
de133ec848850dac516d805bfbc18873e1a53543
left standby.signal in the recovery target node in case when pg_rewind
is used. As a result, the standby node became primary after
restart.
Tatsuo Ishii [Wed, 5 Aug 2020 02:37:28 +0000 (11:37 +0900)]
Fix query cache bug.
If EXPLAIN ANALYZE takes data-modifying SQL, it must invalidate query
cache, but actually it was missed.
Author: Hou, Zhijie
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-August/003767.html
Tatsuo Ishii [Wed, 5 Aug 2020 01:47:06 +0000 (10:47 +0900)]
Fix pgpool_setup problem with -r option and PostgreSQL 12.
pgpool_setup -r uses pg_rewind if it's possible. After rewinding,
pgpool_setup failed to create standby.signal file, which is required in
PostgreSQL 12 or later, on the recovered target node. As a result, the
recovered node booted up as a primary server, rather than a standby
server.
Discussion: https://www.pgpool.net/mantisbt/view.php?id=624
Tatsuo Ishii [Sun, 2 Aug 2020 07:00:41 +0000 (16:00 +0900)]
Implement automatic writing function detection by checking volatile property.
If a function included in SELECT/WITH has volatile property by
checking system catalog, regard it a writing function. This feature is
available only when the black_function_list and the
white_function_list are both empty. The default of the
black_function_list and the white_function_list are now empty string
so that this feature is enabled by default.
If the black_function_list or the white_function_list is not empty,
keep on existing behavior (only consult the black_function_list or the
white_function_list, not access system catalog).
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-July/003730.html
Reviewed-by: Hou, Zhijie
Bo Peng [Fri, 31 Jul 2020 00:45:39 +0000 (09:45 +0900)]
Change PCP UNIX_DOMAIN_PATH of RPM package to "/var/run/postgresql".
Tatsuo Ishii [Tue, 28 Jul 2020 02:36:11 +0000 (11:36 +0900)]
Doc: clarify what the command actual does.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-July/003744.html
Reviewed-by: Bo Peng
Tatsuo Ishii [Mon, 27 Jul 2020 10:01:56 +0000 (19:01 +0900)]
Fix Coverity warning.
CID
1430580: Null pointer dereferences (REVERSE_INULL).
Patch provided by Hou, Zhijie.
Bo Peng [Mon, 27 Jul 2020 02:40:25 +0000 (11:40 +0900)]
Rename src/redhat/pgpool_rhel7.sysconfig to src/redhat/pgpool_rhel.sysconfig to make this file available on RHEL8/CentOS8.
Tatsuo Ishii [Sun, 26 Jul 2020 01:04:36 +0000 (10:04 +0900)]
Enhance the way getting function names in multiple places.
With query cache enabled, Pgpool-II examines function calls in
SELECTs. However if a function were called with schema qualification,
it was not recognized. This commit fix to allow schema qualifications
in the case.
Also we did not allow schema qualified function names in
black_function_list and white_function_list. The limitation is also
fixed in this commit. Note that if you want to register schema
qualified function names, you have to register function names without
schema qualification as well.
Patch contributed by Hou, Zhijie, Reviewed by me.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-July/003718.html
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-July/003732.html
Bo Peng [Wed, 22 Jul 2020 01:23:13 +0000 (10:23 +0900)]
Fix typos.
Patch is created by Lu, Chenyang.
Muhammad Usama [Tue, 21 Jul 2020 09:14:00 +0000 (14:14 +0500)]
New feature: dml dml object level load balance support
Patch contributed by Sunbiao<sunbiao@highgo.com> and
some enhancements done by me
Tatsuo Ishii [Fri, 17 Jul 2020 01:02:51 +0000 (10:02 +0900)]
Doc: fix explanation about how to decide load balance node.
It should have mentioned about statement_level_load_balance.
Tatsuo Ishii [Thu, 16 Jul 2020 10:46:33 +0000 (19:46 +0900)]
Doc: fix typo in memqcache doc.
Patch contributed by Hou, Zhijie.
Discussion: https://www.pgpool.net/pipermail/pgpool-general/2020-July/007204.html
Tatsuo Ishii [Tue, 14 Jul 2020 12:56:14 +0000 (21:56 +0900)]
Prevent data modifying CTE to be cached.
Data modifying CTE was mistakenly treated as normal read only CTE and
result query was created. As a result subsequent CTE was not
executed.
Problem reported and patch created by Hou, Zhijie.
Subtle changes to the regression test by me.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-July/003705.html
Bo Peng [Fri, 10 Jul 2020 02:37:22 +0000 (11:37 +0900)]
Doc: update "Installation from RPM" section.
Tatsuo Ishii [Thu, 9 Jul 2020 00:11:03 +0000 (09:11 +0900)]
Fix pgpool hang in a corner case.
It is possible that an "out of band" message from backend has been
read into buffer at the time when a ready for query message is
processed. If the messages are from all backends, there should be no
problem because ProcessBackendResponse() will read the messages from
all backends by using read_kind_from_backend(). However there could be
a corner case: 1) If the message is coming from only one of backend
(this could happen when recovery conflict or backend receiving SIGTERM
and so on) and 2) the message is already in the backend read
buffer. In this case pgpool will hang in pool_read() called by
read_kind_from_backend() at either: 1)
read_kind_from_one_backend(frontend, backend, (char *) &kind,
MASTER_NODE_ID) (the message is not coming from master backend) or 2)
pool_read(CONNECTION(backend, i), &kind, 1) (the message is not coming
from other than master).
Note If the message is not in the buffer, there should be no problem
since read_packets_and_process() will take care that "out of band"
messages.
The solution is, read and discard such a message in ReadyforQuery(),
emitting log to make sure that the read buffer is empty after
returning from ReadyForQuery(). (remember that unless the ready for
query message is returned to frontend, the frontend will not issue
next query and there's should be no response from backend except the
out of band messages).
If the message was FATAL, the backend will disconnect to pgpool. So
next time pgpool should notice that the connection is closed anyway.
For the master branch, probably we should treat that kind of FATAL
message in a same way as read_packets_and_process() already does. This
requires some code refactoring and I would like to leave the job
separated from this commit.
Tatsuo Ishii [Wed, 8 Jul 2020 01:25:12 +0000 (10:25 +0900)]
Fix per_node_error_log() so that it respects unread parameter.
per_node_error_log() unconditionally set the "unread" parameter to
true when it should have respected the unread parameter given by user.
It seems all callers to per_node_error_log() set the unread parameter
to true anyway. The only exception is pool_do_auth. As far as testing
test 03[0-4] regression tests, the change is ok.
In summary this change will not affect to Pgpool-II but bug is bug.
Bo Peng [Tue, 7 Jul 2020 04:49:07 +0000 (13:49 +0900)]
Fix coverity SSL memory leak errors.
Tatsuo Ishii [Tue, 7 Jul 2020 03:35:10 +0000 (12:35 +0900)]
Fix lazy coding of ereport in pgpool_adm extension.
Before it was coded that ereport's "domain" parameters were just "0".
This generated lots of compiler warning. To fix this give plausible
error codes to each ereport call.
Tatsuo Ishii [Mon, 6 Jul 2020 23:32:22 +0000 (08:32 +0900)]
Allow to use argument names in pgpool_adm functions.
Now pgpool_adm functions allow to use named parameters like:
SELECT * FROM pcp_node_count(host => 'localhost', port => 11001, username => 't-ishii', password => 't-ishii');
Patch provided by Hou, Zhijie, minor tweak by me.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-July/003687.html
Tatsuo Ishii [Fri, 3 Jul 2020 10:44:57 +0000 (19:44 +0900)]
Change pgpool_setup so that log_line_prefix includes application name.
This makes analyzing of pgpool log since each line includes process
name. Also tweak regression test 008 and 009 so that they do not fail
because of extra logging item.
Tatsuo Ishii [Fri, 3 Jul 2020 10:37:26 +0000 (19:37 +0900)]
Unbreak 010.rewrite_timestamp test.
commit
35037bcd837472f04b92008362db7f3cefa88bec added inclusion of
src/main/pool_globals.c, which caused duplication of
get_application_name() in
src/test/regression/tests/010.rewrite_timestamp/timestamp/main.c.
Tatsuo Ishii [Fri, 3 Jul 2020 02:20:12 +0000 (11:20 +0900)]
Doc: add logging collector Japanese doc.
Muhammad Usama [Thu, 2 Jul 2020 18:05:30 +0000 (23:05 +0500)]
Fix for log messages from child processes printing the main process's PID
The issue was caused by my log collector commit. And the reason was: as part of
the log collector implementation, I added a new global variable myProcPid to hold
to the process PID so that we can avoid calling getpid() function every time
during the construction of the log message.
The change was meant to increase the performance but unfortunately I missed
setting the myProcPid variable after fork() at one of the places.
(After forking the pgpool child process).
So to make sure we can avoid such mistakes in the future, I also did a little bit
of refactoring as part of this fix and created a new function that sets all
process-related global variables (process_application_name, myProcPid and
processType) so that we have a single place to add/modify such stuff.
Apart from that, I have made a few changes in functions for setting the
process_application_name and moved them to pool_globasl.c from pgpool_main.c
Muhammad Usama [Thu, 2 Jul 2020 11:32:38 +0000 (16:32 +0500)]
Fix coverity warnings
CID-
1429985 and CID-
1429986
Tatsuo Ishii [Thu, 2 Jul 2020 07:49:20 +0000 (16:49 +0900)]
Doc: allow units to be specified with configuration settings.
Tatsuo Ishii [Thu, 2 Jul 2020 07:12:20 +0000 (16:12 +0900)]
Change unit used in authentication_timeout.
Change it from 60 to 1min.
Tatsuo Ishii [Thu, 2 Jul 2020 05:51:47 +0000 (14:51 +0900)]
Add public API to invalidate query cache by table/database oid.
This would be called by new PCP command to invalidate particular query
cache.
Tatsuo Ishii [Wed, 1 Jul 2020 02:33:04 +0000 (11:33 +0900)]
Let pgpool_setup set REPEATABLE READ while creating snapshot isolation mode.
030.snapshot_isolation/test.sh does not need the work anymore.
Tatsuo Ishii [Mon, 29 Jun 2020 23:07:12 +0000 (08:07 +0900)]
Doc: fix typo in the explanation of pgpool_adm.
Patch provided by sherlockcpp.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-June/003679.html
Tatsuo Ishii [Mon, 29 Jun 2020 22:57:28 +0000 (07:57 +0900)]
Doc: fix illegal character in failover.sgml.
Muhammad Usama [Mon, 29 Jun 2020 12:23:40 +0000 (17:23 +0500)]
Documentation updates for log collector
Tatsuo Ishii [Mon, 29 Jun 2020 00:35:23 +0000 (09:35 +0900)]
Fix typo in query cache module.
Tatsuo Ishii [Sat, 27 Jun 2020 08:13:55 +0000 (17:13 +0900)]
Allow to use snapshot isolation mode with extended query protocol.
Also 030 regression test now tests extended query case by using
pgbench -M extended as well.
Muhammad Usama [Fri, 26 Jun 2020 19:08:24 +0000 (00:08 +0500)]
Adding log collector
Molding the syslogger of PostgreSQL to work with
Pgpool-II so that we can have log rotation
Documentation updates will follow soon.
Tatsuo Ishii [Fri, 26 Jun 2020 11:46:33 +0000 (20:46 +0900)]
Fix startup packet reading process.
If there are parameters other than "user", "database" and
"application_name", reading startup packet resulted in failing to
extract user: "no PostgreSQL user name specified in startup
packet". This was found while a frontend was trying to speak
replication protocol by using "replication=database".
Discussion: https://www.pgpool.net/pipermail/pgpool-general/2020-June/007162.html
Takuma Hoshiai [Fri, 26 Jun 2020 08:29:44 +0000 (17:29 +0900)]
Add support for LDAP authentication between client and pgpool.
This commit supports LDAP authentication between application and Pgpool.
LDAP is used only to validate the user name/password pairs. LDAP parameter
(such as ldapserver,port, ldapbinddn, etc..) is specified in a row of
pool_hba.conf. When user connection is matched by ldap authentication rows
in pool_hba.conf, access ldapserver and search matching informatinonfor
username/passord.
pgpool has 'backend_use_password' option which is particular parameter
in auth-options. It is able to reuse authentication password which was
used between application and pgpool to postgres authentication.
If we use this feature, Pgpool-II must be build with --with-ldap support.
Tatsuo Ishii [Thu, 25 Jun 2020 23:36:20 +0000 (08:36 +0900)]
Fix load balance in snapshot isolation mode.
Commit:
6f35af7485732bebec33c174ca7276fe5a62fc06
broke the case when SELECT is in a none read only transaction.
Tatsuo Ishii [Thu, 25 Jun 2020 02:24:20 +0000 (11:24 +0900)]
Set application name in pcp process.
This was forgotten in commit:
d8434662d63c115280779941af61252663f9c134
Takuma Hoshiai [Tue, 23 Jun 2020 05:46:38 +0000 (14:46 +0900)]
Doc: add note about auto_failback
If user uses replication_slot, replication slot is may deleted by
failover_command when standby node is down. In this case, pgpool
can't get replication status, so the feature of auto failback is
may not worked.
Tatsuo Ishii [Mon, 22 Jun 2020 10:59:01 +0000 (19:59 +0900)]
Allow to SELECT load balance in snapshot isolation mode.
- In an read only explicit transaction
- If outside transaction, transaction mode is read only
- In other case, check if the query is really read only SELECT.
Also modify 001.load_balance test to check snapshot isolation mode.
Takuma Hoshiai [Mon, 22 Jun 2020 02:45:47 +0000 (11:45 +0900)]
Fix miscount of connection_count
There is a possibility that pgpool duplicate the count down of conections
when query cancel such as pg_terminate_backend() commmand.
Tatsuo Ishii [Sat, 20 Jun 2020 00:36:40 +0000 (09:36 +0900)]
Add regression test for snapshot isolation mode.
To test the mode, we use two concurrent transactions generated by 2
pgbench instances. Tx A updates a row of table t1. Tx B retrieves the
row in t1 and INSERT INTO log table. If atomic visibilty is kept,
after Tx A and B finishes rows of log on node 0 and node 1 should be
identical. In fact in the snapshot isolation mode, they are
identical. On the other hand, in the existing native replication mode,
they are not identical.
Also add missing sample configuration file for the snapshot isolation
mode.
Tatsuo Ishii [Fri, 19 Jun 2020 23:59:57 +0000 (08:59 +0900)]
Fix bugs in snapshot isolation mode.
- Did not call si_commit_request() when starting an internal
transaction. This led to incorrect commit counter in
si_commit_done().
- Fix incorrect preprocessor symbol for DEBUGGING SI module.
Muhammad Usama [Fri, 19 Jun 2020 21:44:50 +0000 (02:44 +0500)]
Allow units to be specified with configuration settings.
The bulk of the code for this feature is stolen from PostgreSQL source
so making the pgpool configuration file syntax more inline with that of PG's
Similarly, PGPOOL SHOW command now outputs units along with value
postgres=# pgpool show search_primary_node_timeout;
search_primary_node_timeout
-----------------------------
5min
(1 row)
Tatsuo Ishii [Fri, 19 Jun 2020 04:41:59 +0000 (13:41 +0900)]
Add new clustering mode "snapshot isolation mode".
Here's the first cut to introduce the Pangea algorythm [1]. I deicded
to add new clustering mode called "snapshot isolation mode" for
this.
[1] http://www.vldb.org/pvldb/vol2/vldb09-694.pdf
What I have done in this commit are:
- Implement Pangea algorythm for r/w transactions.
- Create docs for the new clustering mode.
Please note that to use the new clustering mode, PostgreSQL's default
transaction isolation mode must be "REPEATABLE READ" (not READ
COMMITTED nor SERIALIZABLE). At this point I tend to think that the
new feature should be marked as "experimental" since I am not
convinced that the new mode does not have any defect at the when 4.2
is released.
BTW, I have added new section "Bibliography" to give a credit to
Pangea. However it seems the rendering of it is different from what
PostgreSQL has. Can someone please help?
"Takeshi Mishima and Hiroshi Nakamura, "Pangea: An Eager Database
Replication Middleware guaranteeing Snapshot Isolation without
modification of Database Servers", Proc. VLDB Conference, Aug. 2009."
I was expecting something like this:
"[mishima2009] Pangea: An Eager Database Replication Middleware
guaranteeing Snapshot Isolation without modification of Database
Servers", "Takeshi Mishima and Hiroshi Nakamura, Proc. VLDB
Conference, Aug. 2009."
What I have not done are:
(1) Check if the transaction is read only and if so, then load balance
it.
(2) regression test
(3) How to deal with SERIAL/sequence.
(4) Deal with extended query protocol
For (1), maybe we could have following rules:
- If it's a read only SELECT and it's not inside an explicit
transaction, we can load balance it.
- If it's a read only SELECT and it's inside an explicit transaction,
we can not load balance it.
The latter is different from native replication mode.
For (2) maybe we need new testing framework to give a control the
timing to give queries to backend. If you have an idea, please let me
know.
For (3) I have no idea. Please help.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-April/003580.html
Tatsuo Ishii [Fri, 5 Jun 2020 00:12:09 +0000 (09:12 +0900)]
Doc: add an untranslated sentence in auto_failback.
Tatsuo Ishii [Thu, 4 Jun 2020 03:47:01 +0000 (12:47 +0900)]
Doc: Japanese documentation for new pcp_reload_config utility.
Patch contributed by "Jianshen Zhou <zhoujianshen@highgo.com>".
Translated into Japanese by me.
Muhammad Usama [Wed, 3 Jun 2020 16:28:50 +0000 (21:28 +0500)]
English Documentation for pcp_reload_config utility
Patch contributed by "Jianshen Zhou <zhoujianshen@highgo.com>"
and enhanced by me.
Also, contains a small update for pcp_stop_utility English doc
Takuma Hoshiai [Tue, 2 Jun 2020 08:04:30 +0000 (17:04 +0900)]
Fix incorrect execution of failover process
This problem happen by executing pg_terminate_backend command with native replication.
In addition, add regression test for pg_terminate_backend command.
Bo Peng [Mon, 1 Jun 2020 00:04:07 +0000 (09:04 +0900)]
Doc: fix typo.
Tatsuo Ishii [Sat, 30 May 2020 05:27:59 +0000 (14:27 +0900)]
Doc: fix typo in failover document.
Also fix inappropriate xref.
Tatsuo Ishii [Wed, 27 May 2020 06:24:07 +0000 (15:24 +0900)]
Fix 004.watchdog test crash on IBM Z hardware.
When watchdog copies primary node id on the master watchdog node, it
did not consider the case that primary node id on the shared memory
(Req_info->primary_node_id) is remaining in the initial value (-2),
which causes out of range subscript access to backend info
array. Interestingly accessing array[-2] does not crash on intel
architecture but does crash IBM Z hardware. Anyway the reason why the
value remains in -2 is that the regression test is performed in raw
mode. I think the code block handling the primary node id should only
be executed in streaming or logical replication mode.
Bug report and patch provided by gregn123, slightly modified by me.
Mantis bug report: https://www.pgpool.net/mantisbt/view.php?id=614
Tatsuo Ishii [Sun, 24 May 2020 02:35:52 +0000 (11:35 +0900)]
Fix 021.pool_passwd_auth test failure.
In the previous commit, path to pg_enc command was not set.
Tatsuo Ishii [Sat, 23 May 2020 07:42:06 +0000 (16:42 +0900)]
Doc: mention about environment variable PGPOOLKEYFILE in pg_enc command.
pg_enc command actually recognizes the environment variable. The
implementation has been doing so.
Tatsuo Ishii [Sat, 23 May 2020 03:59:47 +0000 (12:59 +0900)]
Enhance scram/md5 auth regression test to use AES256 encrypted password.
021.pool_passwd_auth used plain text password in pool_passwd. AES256
encrypted password is more appropriate for the real world use case.
Tatsuo Ishii [Sat, 23 May 2020 03:17:05 +0000 (12:17 +0900)]
Doc: add explanation about .pgpoolkey.
Also add indexes for .pgpoolkey and PGPOOLKEYFILE.
Bo Peng [Fri, 22 May 2020 07:32:27 +0000 (16:32 +0900)]
Doc: Improve the description of wd_priority.
Patch is provided by Kenichiro Tanaka.
Tatsuo Ishii [Fri, 22 May 2020 01:58:48 +0000 (10:58 +0900)]
Add mention about hostssl/hostnossl to pool_hba.conf sample file.
Although hostssl/hostnossl are supported, pool_hba.conf comment did
not mention about it.
Muhammad Usama [Thu, 21 May 2020 16:32:38 +0000 (21:32 +0500)]
New PCP utility 'pcp_reload_pgpool' for reloading the pgpool configuration
The utility also supports the newly added cluster scope for reloading the
configurations on all Pgpool-II nodes part of the watchdog cluster.
Documentation updates for the utility are still remaining.
Patch contributed by "Jianshen Zhou <zhoujianshen@highgo.com>"
and enhanced by me.
Bo Peng [Mon, 18 May 2020 20:16:27 +0000 (05:16 +0900)]
Doc: Add release-note.
Bo Peng [Mon, 18 May 2020 12:12:25 +0000 (21:12 +0900)]
Change relative path of SSL files to configuration directory.
Patch is created by Umar Hayat and Japanese documentation is added by me.
Bo Peng [Sun, 10 May 2020 00:27:48 +0000 (09:27 +0900)]
Doc: Add note about if_up_cmd and if_down_cmd command.
Tatsuo Ishii [Wed, 6 May 2020 06:48:32 +0000 (15:48 +0900)]
Doc: add --scope option to Japanese pcp_stop_pgpool manual.
Also add mention about the default value of --mode and --scope.
Muhammad Usama [Tue, 5 May 2020 19:30:33 +0000 (00:30 +0500)]
Fix:
0000608: pgpool ssl front end accept all ciphers. not working as expected.
SSL cipher_list needed to be set for server-side SSL context
instead for the client-side context.
Muhammad Usama [Mon, 4 May 2020 11:38:39 +0000 (16:38 +0500)]
New feature: cluster scope for pcp_stop_pgpool utility command
Make pcp_stop_pgpool utility to terminate (optional) all the pgpool nodes in the
watchdog cluster
For that purpose added "--scope=cluster/local" option in the pcp_stop_pgpool
utility to set the scope of the command. The scope could be either
'local' or 'cluster'
Along with that, the commit adds the infrastructure in watchdog for broadcasting
the custom commands over the watchdog network. This will make it easier to
implement the --scope=cluster for other PCP utilities.
Muhammad Usama [Mon, 4 May 2020 11:35:35 +0000 (16:35 +0500)]
Some code reorganization
While working on the pcp_stop_pgpool command to implement cluster mode,
I realized that we have a lot of clutter in a few source files. Especially
pool.h header file. So I also did some code reorganization and added a
bunch of new includefiles in the hope to reduce the size of pool.h and
also get rid of too many global functions.
Along the way, I found out there are few other files that need a little
shredding as well, src/main/pgpool_main.c and src/protocol/child.c are at
the top of this wanted to be trimmed list. So as part of this commit.
I have taken a few things related to internal commands
(interface between the child processes can and Pgpool-II main process)
from pgpool_main.c file and moved them into new "src/main/pool_internal_comms.c"
Similarly, src/protocol/child.c had and still has so many functions that do
not fit with the personality of child.c file. So I have moved some of the
stuff related to DB functions info src/protocol/pool_pg_utils.c file.
Having done that I think this is still not enough and we may require another
round ( if someoneis willing to work on it) of a source file reorganization.
Tatsuo Ishii [Mon, 4 May 2020 03:46:53 +0000 (12:46 +0900)]
Fix extended query tests.
Commit https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=
eee4c86c03546abe4e33396c808c8425c23a443c
broke extra scripts because it added Bind message log if log_per_node_statement is enabled.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-May/003611.html
Takuma Hoshiai [Tue, 28 Apr 2020 08:50:19 +0000 (17:50 +0900)]
Improve process status of extend query
Previously, pgpool status is incorrect with extend query.
For example, when query is executing, but process status
is idle. This fix can display process status correctly.
If 'Execute' message is received, pgpool display process
status the same as Simple query.
For example:
user databse SELECT
In addition, fix log_statement bug. Previously, statement
using extend query is not logged by log_statement parameter.
It was improved in this commit.
Other, log_per_node_statement can log query with BIND message.
Tatsuo Ishii [Sun, 26 Apr 2020 02:20:06 +0000 (11:20 +0900)]
Downgrade too verbose authentication logs.
When frontend authentication succeeds with clear text password and md5
authentication, message is out with log priority LOG. This floods
Pgpool-II log on busy systems. To fix this, downgrade the LOG message
to DEBUG1. Note that CERT auth related log level is still
LOG. Downgrading it causes regression 024 and 029 failure.
Discussion:
https://www.pgpool.net/mantisbt/view.php?id=606
https://www.pgpool.net/pipermail/pgpool-general/2020-April/007064.html
Tatsuo Ishii [Sun, 26 Apr 2020 00:58:50 +0000 (09:58 +0900)]
Remove duplicate global data.
This causes a link error on some system (Fedra 32).
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-April/003593.html
Tatsuo Ishii [Thu, 23 Apr 2020 12:32:28 +0000 (21:32 +0900)]
Fix unnecessary checks.
Patch contributed by sherlockcpp.
Discussion: https://www.pgpool.net/pipermail/pgpool-general/2020-April/007062.html
Tatsuo Ishii [Thu, 23 Apr 2020 05:32:28 +0000 (14:32 +0900)]
Doc: fix typo.
Patch contributed by Umar Hayat.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2020-April/003587.html
Tatsuo Ishii [Mon, 20 Apr 2020 05:25:09 +0000 (14:25 +0900)]
Forgot to push file in previous commit.
Previous commit: https://git.postgresql.org/gitweb?p=pgpool2.git;a=commitdiff;h=
589a81960939102fb620b395b8cd45434d6aa4bc
Tatsuo Ishii [Sat, 18 Apr 2020 04:03:44 +0000 (13:03 +0900)]
Fix unnecessary free pointed out by Coverity.
Since buf is already checked whether it is NULL or not, it is not
necessary to check the variable before freeing it.
Tatsuo Ishii [Sat, 18 Apr 2020 03:50:47 +0000 (12:50 +0900)]
Fix array access overrun pointed out by Coverity.