Tatsuo Ishii [Thu, 20 Jun 2024 06:44:18 +0000 (15:44 +0900)]
Fix typo in comment.
Bo Peng [Wed, 19 Jun 2024 06:19:30 +0000 (15:19 +0900)]
Doc: add the missing default values for virtual IP related parameters.
Tatsuo Ishii [Fri, 14 Jun 2024 00:30:46 +0000 (09:30 +0900)]
Fix "show pool_processes" to not show row description twice.
processes_reporting() accidentaly called both send_row_description()
and send_row_description_and_data_rows().
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2024-June/004472.html
[pgpool-hackers: 4471] [PATCH] printing empty row first in query "show pool_process"
Author: Kwangwon Seo
Back patch to V4.2 where the problem started.
Tatsuo Ishii [Tue, 11 Jun 2024 11:15:08 +0000 (20:15 +0900)]
Fix segfault in a child process.
It is reported that pgpool child segfaulted [1].
[snip]
In the down thread it is reported that despite VALID_BACKEND(i)
returns true, backend->slots[i] is NULL, which should have been filled
by new_connection().
It seems there's a race condition. In new_connection(), there's a code
fragment:
/*
* Make sure that the global backend status in the shared memory
* agrees the local status checked by VALID_BACKEND. It is possible
* that the local status is up, while the global status has been
* changed to down by failover.
*/
A--> if (BACKEND_INFO(i).backend_status != CON_UP &&
BACKEND_INFO(i).backend_status != CON_CONNECT_WAIT)
{
ereport(DEBUG1,
(errmsg("creating new connection to backend"),
errdetail("skipping backend slot %d because global backend_status = %d",
i, BACKEND_INFO(i).backend_status)));
/* sync local status with global status */
B--> *(my_backend_status[i]) = BACKEND_INFO(i).backend_status;
continue;
}
It is possible that at A backend_status in the shared memory is down
but by the time it reaches B the status has been changed to up. And
new_connection() skipped to create a backend connection. This seems to
explain why the connection slot is NULL while VALID_BACKEND returns
true. To prevent the race condtion, backend_status in shared memory is
copied to a local variable and evaluate it. Also the VALID_BACKEND
just before:
pool_set_db_node_id(CONNECTION(backend, i), i);
is changed to:
if (VALID_BACKEND(i) && CONNECTION_SLOT(backend, i))
so that it prevents crash just in case.
[1] [pgpool-general: 9104] Another segmentation fault
Muhammad Usama [Tue, 11 Jun 2024 06:52:00 +0000 (11:52 +0500)]
Fix: 823: Watchdog dies and kills pgpool2 when network gets shortly interrupted.
With network monitoring enabled, a Pgpool node would shut down immediately if it
lost all network interfaces or assigned IP addresses, providing extra protection
by quickly removing a non-communicative node from the cluster.
The issue was that Pgpool responded to network blackout events even when network
monitoring was disabled. This fix ensures that the network monitoring socket is
not opened when network monitoring is not enabled, preventing unnecessary shutdowns.
Tatsuo Ishii [Mon, 10 Jun 2024 02:23:47 +0000 (11:23 +0900)]
Fix "pgpool reset" command not working if watchdog is enabled.
[pgpool-hackers: 4465] abnormal behavior about PGPOOL RESET. and proposal a patch file.
reported that "pgpool reset" command fails if watchdog is enabled.
test=# PGPOOL RESET client_idle_limit;
SET
ERROR: Pgpool node id file �y/pgpool_node_id does not exist
DETAIL: If watchdog is enable, pgpool_node_id file is required
message type 0x5a arrived from server while idle
message type 0x43 arrived from server while idle
message type 0x5a arrived from server while idle
SetPgpoolNodeId() tried to obtain the path to the node id file by
using global variable config_file_dir and failed because it points to
an automatic variable in ParseConfigFile().
To fix this, change the config_file_dir from a pointer to an array and
save the path string into config_file_dir in ParseConfigFile().
Also regression test is added to 004.watchdog.
Bug reported and problem analysis by keiseo.
Back patch to V4.2 in which the node id file was introduced.
Tatsuo Ishii [Fri, 7 Jun 2024 10:39:53 +0000 (19:39 +0900)]
Mega typo fix for docs and program source codes.
Author: Umar Hayat
Tatsuo Ishii [Tue, 4 Jun 2024 01:11:05 +0000 (10:11 +0900)]
Fix psql_scan crash.
It was reported that psql_scan crashes while determining whether a
string in a long query is psql variable (i.e. starting with ":") or
not.
https://github.com/pgpool/pgpool2/issues/54
This is because callback struct were not provided while calling
psql_scan_create(). Later psql_scan() tries to invoke a callback and
crashes because the pointer to the callback struct is NULL. To fix
this, provide PsqlScanCallbacks struct with a NULL pointer inside to
the callback function. With this, psql_scan() avoids to invoke a
callback.
Backpatch to master, V4.5, V4.4, V4.3, V4.2 and V4.1 where psql_scan
was introduced.
Bo Peng [Mon, 20 May 2024 05:18:35 +0000 (14:18 +0900)]
Delete unnecessary if branch.
https://github.com/pgpool/pgpool2/issues/52
Bo Peng [Thu, 16 May 2024 01:00:22 +0000 (10:00 +0900)]
Doc: update Copyright.
Bo Peng [Tue, 14 May 2024 23:49:44 +0000 (08:49 +0900)]
Prepare 4.2.17
Bo Peng [Tue, 14 May 2024 23:31:16 +0000 (08:31 +0900)]
Doc: Bump version to 4.2.17
Bo Peng [Tue, 14 May 2024 22:50:16 +0000 (07:50 +0900)]
Doc: add release notes.
Bo Peng [Tue, 7 May 2024 00:20:54 +0000 (09:20 +0900)]
Doc: fix documentation typos.
Bo Peng [Wed, 1 May 2024 06:42:10 +0000 (15:42 +0900)]
Fixed compiler error with -Werror=implicit-function-declaration
- Add missing header files in autoconf check and
- Add LDAP_DEPRECATED to include prototypes for deprecated ldap functions
Patch is created by Vladimir Petko.
Tatsuo Ishii [Thu, 4 Apr 2024 11:27:50 +0000 (20:27 +0900)]
Fix assorted causes of segmentation fault.
It is reported that pgpool and its child process segfault in certain
cases when failover involved.
Hhealth check has an issue with connection "slot" memory. It is
managed by HealthCheckMemoryContext. slot is the pointer to the
memory. When elog(ERROR) is raised, pgpool long jumps and resets the
memory context. Thus, slot remains as a pointer to freed memory. To
fix this, always set NULL to slot right after the
HealthCheckMemoryContext call.
Similar issue is found with streaming replication check too and is
also fixed in this commit.
Problem reported and analyzed: Emond Papegaaij
Backpatch-through: v4.3-v4.1
Discussion:
[pgpool-general: 9070] Re: Segmentation after switchover
Tatsuo Ishii [Wed, 3 Apr 2024 10:13:53 +0000 (19:13 +0900)]
Fix uninitialized memory error.
It was reported that valgrind found several errors including an
uninitialized memory error in read_startup_packet. It allocates memory
for user name in a startup packet in case cancel or SSL request using
palloc, and later on the memory is used by pstrdup. Since memory
allocated by palloc is undefined, this should have been palloc0.
Bug reported by: Emond Papegaaij
Backpatch-through: v4.1
Discussion:
[pgpool-general: 9065] Re: Segmentation after switchover
https://www.pgpool.net/pipermail/pgpool-general/2024-April/009126.html
Bo Peng [Wed, 3 Apr 2024 07:18:17 +0000 (16:18 +0900)]
Doc: enhance "Upstream server connection" documentation.
Tatsuo Ishii [Tue, 2 Apr 2024 10:30:27 +0000 (19:30 +0900)]
Fix errors/hung up when load_balance_mode is off.
Commit:
3f3c1656 Fix statement_level_load_balance with BEGIN etc.
brought errors/hung up when load_balance_mode is off, primary node id
is not 0 and queries are BEGIN etc.
pool_setall_node_to_be_sent() checked if the node is primary. If not,
just returned with empty where_to_send map which makes
set_vrtual_main_node() not to set
query_context->virtual_main_node_id. As a result, MAIN_NODE macro
(it's actually pool_virtual_main_db_node_id()) returns
REAL_MAIN_NODE_ID, which is 0 if node 0 is alive (this should have
been primary node id).
Following simple test reveals the bug.
(1) create a two-node cluster using pgpool_setup
(2) shutdown node 0 and recover node 1 (pcp_recovery_node 0). This
makes node 0 to be standby, node 1 to be primary.
(3) add followings to pgpool.conf and restart whole cluster.
load_balance_mode = off
backend_weight1 = 0
(4) type "begin" from psql. It gets stuck.
Bug found and analyzed by Emond Papegaaij.
Discussion: https://www.pgpool.net/pipermail/pgpool-general/2024-March/009113.html
Backpatch-through: v4.1
Masaya Kawamoto [Thu, 28 Mar 2024 00:16:55 +0000 (00:16 +0000)]
Doc: language cleanup in Japanese document
Replace "マスター" with "プライマリ"
Tatsuo Ishii [Mon, 25 Mar 2024 07:15:50 +0000 (16:15 +0900)]
Fix compile errors with certain CFLAGS.
https://github.com/pgpool/pgpool2/issues/42 reported that with CFLAGS
-flto=4 -Werror=odr -Werror=lto-type-mismatch -Werror=strict-aliasing
gcc emits errors. Some of them are mistakes when their sources were
brought in from PostgreSQL. This commit fixes them. Note that I was
not able to suppress some errors at least with my gcc (9.4.0). This
may be because gcc bug (false positives) or just a bug with the old
gcc, I don't know at this point. Maybe someday revisit this.
Discussion:
[pgpool-hackers: 4442] Fixing GitHub issue 42
https://www.pgpool.net/pipermail/pgpool-hackers/2024-March/004443.html
../src/include/query_cache/pool_memqcache.h:251:20: warning: type of 'pool_fetch_from_memory_cache' does not match original declaration [-Wlto-type-mismatch]
251 | extern POOL_STATUS pool_fetch_from_memory_cache(POOL_CONNECTION * frontend,
| ^
query_cache/pool_memqcache.c:731:1: note: 'pool_fetch_from_memory_cache' was previously declared here
731 | pool_fetch_from_memory_cache(POOL_CONNECTION * frontend,
| ^
query_cache/pool_memqcache.c:731:1: note: code may be misoptimized unless '-fno-strict-aliasing' is used
../src/include/utils/palloc.h:64:22: warning: type of 'CurrentMemoryContext' does not match original declaration [-Wlto-type-mismatch]
64 | extern MemoryContext CurrentMemoryContext;
| ^
../../src/utils/mmgr/mcxt.c:40:15: note: 'CurrentMemoryContext' was previously declared here
../../src/utils/mmgr/mcxt.c:40:15: note: code may be misoptimized unless '-fno-strict-aliasing' is used
../src/include/utils/memutils.h:55:22: warning: type of 'TopMemoryContext' does not match original declaration [-Wlto-type-mismatch]
55 | extern MemoryContext TopMemoryContext;
| ^
../../src/utils/mmgr/mcxt.c:46:15: note: 'TopMemoryContext' was previously declared here
../../src/utils/mmgr/mcxt.c:46:15: note: code may be misoptimized unless '-fno-strict-aliasing' is used
../src/include/pool_config.h:646:22: warning: type of 'pool_config' does not match original declaration [-Wlto-type-mismatch]
646 | extern POOL_CONFIG * pool_config;
| ^
config/pool_config.l:46:14: note: 'pool_config' was previously declared here
46 | POOL_CONFIG *pool_config = &g_pool_config; /* for legacy reason pointer to the above struct */
| ^
config/pool_config.l:46:14: note: code may be misoptimized unless '-fno-strict-aliasing' is used
Bo Peng [Mon, 25 Mar 2024 07:39:12 +0000 (16:39 +0900)]
Fixed comments in sample pgpool.conf.
- The comment for sr_check_period. The default value should be 10 seconds.
- Also fixed some typos in comments.
Patch is created by hiroin and modified by Bo Peng.
Tatsuo Ishii [Wed, 20 Mar 2024 22:31:29 +0000 (07:31 +0900)]
Allow reset queries to run even if extended queries do not end.
Commit
240c668d "Guard against inappropriate protocol data." caused
reset queries fail if extended query messages do not end. This commit
fix that by checking whether we are running reset queries in
SimpleQuery(). Also add the test case for this.
Tatsuo Ishii [Mon, 18 Mar 2024 01:33:16 +0000 (10:33 +0900)]
Guard against inappropriate protocol data.
If a simple query message arrives before a sequence of extended query
messages ends (that is, no sync message arrives or some ready for
query messages corresponding the sync message do not arrive yet),
pgpool could hang. This is because the query context in the session
context for the simple query is overwritten by the query contexts of
the extended query messages.
This commit implements a guard in SimpleQuery() by checking whether
extended query protocol messages ended. If they do not end, raise a
FATAL error. A known example detected by this checking is JDBC
driver's "autosave=always" option. This means pgpool will not accept
the option after this commit until the issue (sending a simple
protocol message before ending extended query message protocol) is
fixed by the JDBC driver side.
Discussion:
[pgpool-hackers: 4427] Guard against ill mannered frontend
https://www.pgpool.net/pipermail/pgpool-hackers/2024-February/004428.html
Bo Peng [Sat, 16 Mar 2024 13:11:23 +0000 (22:11 +0900)]
Test: fix test failure introduced in the previous commit.
Bo Peng [Fri, 15 Mar 2024 04:54:59 +0000 (13:54 +0900)]
Test: fixed regression test 005.jdbc to avoid Java 21 deprecation warnings.
Replace Runtime.exec(String) with Runtime.exec(String[]) to avoid Java 21 deprecation warnings.
Patch is created by Vladimir Petko and modified by Bo Peng.
Bo Peng [Thu, 29 Feb 2024 05:11:17 +0000 (14:11 +0900)]
Doc: update copy right.
Bo Peng [Tue, 27 Feb 2024 08:36:36 +0000 (17:36 +0900)]
Prepare 4.2.16
Bo Peng [Tue, 27 Feb 2024 02:34:28 +0000 (11:34 +0900)]
Doc: update document version.
Bo Peng [Tue, 27 Feb 2024 02:28:25 +0000 (11:28 +0900)]
Doc: add release notes.
Bo Peng [Tue, 27 Feb 2024 00:55:28 +0000 (09:55 +0900)]
Remove restore_command from sample scripts.
Restore_command is not required because replication slot is enabled.
It causes occasional failover failure.
Bo Peng [Mon, 26 Feb 2024 10:49:31 +0000 (19:49 +0900)]
Fix the default value of recovery_user.
Tatsuo Ishii [Mon, 26 Feb 2024 07:05:31 +0000 (16:05 +0900)]
Doc: fix Japanese watchdog document.
It mistakenly used "master" watchdog node instead of "leader".
Tatsuo Ishii [Mon, 26 Feb 2024 07:01:27 +0000 (16:01 +0900)]
Doc: fix English watchdog document.
It mistakenly used "main" PostgreSQL node instead of "primary".
Tatsuo Ishii [Sat, 10 Feb 2024 02:50:28 +0000 (11:50 +0900)]
Fix statement_level_load_balance with BEGIN etc.
When statement_level_load_balance is enabled,
BEGIN/END/COMMIT/ABORT/SET/SAVEPOINT/RELEASE SAVEPOINT/DEALLOCATE
ALL/DISCARD were sent to primary node and all standby nodes even if
load_balance_mode is off. This is not only plain wrong but caused slow
down if one of the standby nodes are in remote network. Fix this in
that pgpool sends such queries to primary node only when
load_balance_mode is off.
Note that if load_balance_mode is on and statement_level_load_balance
is on, such queries are sent to all nodes as before. This is
necessary. For example, suppose there are 2 PostgreSQL nodes 0 and
1. An explicit transaction starts followed by two read only
SELECTs. The first SELECT is sent to node 0 because the node 0 is
chosen as the load balance node. The second SELECT is sent to node 1
because the node 1 is chosen as the load balance node. If pgpool has
not sent BEGIN to both node 0 and 1 when the transaction started, the
first or the second SELECT will be executed outside the transaction,
which is not an expected behavior. However this may bring slow down
mentioned above. I guess this has been less known to users and I
decided to add some notes to the statement_level_load_balance doc.
Reported: [pgpool-general: 8998] https://www.pgpool.net/pipermail/pgpool-general/2024-January/009059.html
Discussion: [pgpool-hackers: 4422] https://www.pgpool.net/pipermail/pgpool-hackers/2024-February/004423.html
Backpatch-through: v4.1
Bo Peng [Tue, 9 Jan 2024 03:11:48 +0000 (12:11 +0900)]
Test: now it will be displayed in the log if segfault occurs.
Bo Peng [Wed, 29 Nov 2023 07:08:24 +0000 (16:08 +0900)]
Prepare 4.2.15
Bo Peng [Wed, 29 Nov 2023 06:56:48 +0000 (15:56 +0900)]
Doc: add release notes of 4.0.25-4.4.5.
Bo Peng [Wed, 29 Nov 2023 02:28:35 +0000 (11:28 +0900)]
Update "8.2. Pgpool-II + Watchdog Setup Example" to mention that pcp_listen_addresses = '*' is required to accept all incoming connections.
Bo Peng [Wed, 29 Nov 2023 02:23:50 +0000 (11:23 +0900)]
Comment out "hostname0" in pgpool.conf.
Bo Peng [Fri, 17 Nov 2023 03:46:54 +0000 (12:46 +0900)]
Modify the replication slot name conversion in sample scripts to add support for uppercase hostname.
Patch in created by Sheikh Wasiu Al Hasib and modified by Be Peng.
Tatsuo Ishii [Fri, 6 Oct 2023 21:15:30 +0000 (06:15 +0900)]
Fix breakage in making pgpool_adm.
Since commit
f54efc50 "Fix compiler warnings regarding
DEFAULT_PGSOCKET_DIR.", src/sql/pgpool_adm cannot be compiled with
error:
In file included from pgpool_adm.c:35:0:
../../include/pcp/libpcp_ext.h:32:37: fatal error: parser/pg_config_manual.h: No such file or directory
#include "parser/pg_config_manual.h"
Since pgpool_adm is a PostgreSQL extension module, it should not
include Pgpool-II's pg_config_manual.h. Rather than fixing pgpool_adm
to allow to include pgpool's pg_config_manual.h, modify libpcp_ext.h
so that it does not include pgpool's pg_config_manual.h if PGPOOL_ADM
is defined.
Bo Peng [Fri, 6 Oct 2023 13:47:18 +0000 (22:47 +0900)]
Enhance error messages in escalation.sh.sample.
Bo Peng [Fri, 6 Oct 2023 13:31:33 +0000 (22:31 +0900)]
Fix document typo.
Bo Peng [Fri, 6 Oct 2023 13:30:23 +0000 (22:30 +0900)]
Add SSH timeout in escalation.sh.sample.
Tatsuo Ishii [Wed, 4 Oct 2023 10:24:10 +0000 (19:24 +0900)]
Remove unnecessary logging line.
Debug message was accidentally left.
Tatsuo Ishii [Tue, 3 Oct 2023 01:46:46 +0000 (10:46 +0900)]
Fix compiler warnings regarding DEFAULT_PGSOCKET_DIR.
Commit
3bd6941f accidentally add "#include "pg_config_manual.h", which
should have been "#include parser/pg_config_manual.h". This caused to
include parser/pg_config_manual.h and pg_config_manual.h, which caused
compiler warning:
../src/include/parser/pg_config_manual.h:227:0: warning: "DEFAULT_PGSOCKET_DIR" redefined [enabled by default]
#define DEFAULT_PGSOCKET_DIR "/tmp"
If DEFAULT_PGSOCKET_DIR value was different among these header files.
Reported by: Peng Bo.
Tatsuo Ishii [Thu, 28 Sep 2023 22:46:05 +0000 (07:46 +0900)]
Test: add test case 080 for commit "Fix Describe() so that it does not abort with portal created by DECLARE."
Tatsuo Ishii [Thu, 28 Sep 2023 22:29:28 +0000 (07:29 +0900)]
Fix Describe() so that it does not abort with portal created by DECLARE.
When DECLARE foo CURSOR FOR is executed, a portal named "foo" is
automatically created by PostgreSQL. As the portal is not managed by
Pgpool-II, "Describe foo" message failed with "unable to execute
Describe. unable to get the bind message" error.
To fix this, make Describe() creates a dummy write query context so
that the describe message is sent to primary (streaming replication
mode) or all nodes (replication/snapshot isolation mode).
Problem analysis by Heather Lapointe.
Discussion: https://www.pgpool.net/pipermail/pgpool-general/2023-September/008995.html
Tatsuo Ishii [Wed, 20 Sep 2023 06:07:26 +0000 (15:07 +0900)]
Remove duplication definitions of NAMEDATALEN.
The definition for NAMEDATALEN is now in
src/include/pg_config_manual.h only. Also replace POOL_NAMEDATALEN
with NAMEDATALEN in src/utils/pool_select_walker.c.
I tried to eliminate copying pool_config_manual.h from PostgreSQL but
it seems it is a little bit difficult to do that at this moment.
There are some definitions like pg_attribute_format_arg were added to
it.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2023-August/004366.html
Tatsuo Ishii [Mon, 21 Aug 2023 07:05:40 +0000 (16:05 +0900)]
Doc: enhance follow primary command manual.
Mention that it is necessary to run pcp_attach_node if
pcp_recovery_node is not used.
Tatsuo Ishii [Mon, 21 Aug 2023 02:02:09 +0000 (11:02 +0900)]
Test: enhance pgpool_setup.
Enhance failover script generation and follow primary script
generation so that they create better logging.
Bo Peng [Wed, 16 Aug 2023 05:40:46 +0000 (14:40 +0900)]
Prepare 4.2.14
Bo Peng [Wed, 16 Aug 2023 05:37:15 +0000 (14:37 +0900)]
Doc: Bump version to 4.2.14
Chen Ningwei [Tue, 15 Aug 2023 03:07:13 +0000 (12:07 +0900)]
Doc: add release notes.
Tatsuo Ishii [Wed, 9 Aug 2023 02:10:28 +0000 (11:10 +0900)]
Fix covery warning.
Fix query cache module assigned time_t value to int32 variable. Change
the variable type to int64. Per Coverity. Also use difftime() to
calculate time_t difference. This is a recommended practice.
https://www.jpcert.or.jp/sc-rules/c-msc05-c.html
Bo Peng [Fri, 4 Aug 2023 05:13:05 +0000 (14:13 +0900)]
Fix find_primary_node_repeatedly doesn't terminate within search_primary_node_timeout.
Bo Peng [Thu, 3 Aug 2023 07:57:45 +0000 (16:57 +0900)]
Doc: fix the missing "logger" application name.
Tatsuo Ishii [Tue, 1 Aug 2023 08:20:49 +0000 (17:20 +0900)]
Remove non-standard "//" comments.
Tatsuo Ishii [Tue, 1 Aug 2023 02:49:26 +0000 (11:49 +0900)]
Doc: clarify the meaning of stop mode.
Bo Peng [Wed, 26 Jul 2023 03:55:06 +0000 (12:55 +0900)]
Doc: remove the incorrect information from release note 4.2.10.
Remove "Fix pgpool_recovery extension script. (Tatsuo Ishii)" from release note 4.2.10.
Muhammad Usama [Sun, 23 Jul 2023 17:34:05 +0000 (22:34 +0500)]
Install system exit callback only after initialization of shared memory.
When the on-exit callback gets called because of a failure to acquire
shared memory. The cleanup function can produce a segfault while accessing
process_info, that lives in shared memory.
Although we can also fix this by bailing out from the exit callback when
process_info is NULL but installing the function after successful initialization
of shared memory is a better approach as the rest of the system always assumes
the process_info can never be NULL, and also, there is nothing to
clean up before child processes are spawned.
Tatsuo Ishii [Mon, 17 Jul 2023 10:27:43 +0000 (19:27 +0900)]
Fix crash when v2 protocol is used.
When read_kind_from_backend() accumulates statistics data, it was not
prepared for v2 protocol case.
Bug report and patch from MCanivez.
https://www.pgpool.net/mantisbt/view.php?id=807
Slightly modified by me.
Tatsuo Ishii [Wed, 12 Jul 2023 07:35:46 +0000 (16:35 +0900)]
Fix PREPARE in multi-statement case.
If multi-statement query includes PREPARE in the second or latter
position, and subsequent bind message uses the prepared statement, it
fails with "unable to bind" error because the prepared statement is not
saved in sent messages.
To fix this if such a case found after parsing the statement, create a
query context for the named statement and add it to the sent message
list.
Discussion: https://www.pgpool.net/pipermail/pgpool-general/2023-July/008931.html
For this new regression test 079..multi_prepare is added.
Bo Peng [Tue, 11 Jul 2023 06:10:20 +0000 (15:10 +0900)]
Doc: Enhance SSH public key authentication setting section in "8.2. Pgpool-II + Watchdog Setup Example".
Tatsuo Ishii [Sun, 9 Jul 2023 10:09:26 +0000 (19:09 +0900)]
Fix pgproto to work with bind message using params.
Previously pgproto can only process bind messages without params.
Tatsuo Ishii [Thu, 29 Jun 2023 02:27:31 +0000 (11:27 +0900)]
Doc: fix load balance mode chapter.
In some places "streaming replication mode" was written as "native
replication mode". Also enhance description regarding additional
requirements for load balancing in the streaming replication mode.
Add some indexes.
Tatsuo Ishii [Wed, 28 Jun 2023 02:56:29 +0000 (11:56 +0900)]
Doc: fix load balance mode chapter.
In some places "streaming replication mode" was written as "native
replication mode". Also enhance description regarding additional
requirements for load balancing in the streaming replication mode.
Add some indexes.
Bo Peng [Tue, 27 Jun 2023 02:46:17 +0000 (11:46 +0900)]
Doc: enhance online recovery documentation.
Mention that 2nd stage of online recovery does not work properly only for multiple pgpool nodes without watchdog enabled.
Tatsuo Ishii [Mon, 12 Jun 2023 00:46:47 +0000 (09:46 +0900)]
Add schema qualification to some system catalog inquiry functions.
There were a few places where schema qualification were not used while
issuing system catalog inquiry.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2023-June/004346.html
Bo Peng [Tue, 30 May 2023 13:17:33 +0000 (22:17 +0900)]
Fix sample script log messages.
Tatsuo Ishii [Mon, 22 May 2023 07:27:40 +0000 (16:27 +0900)]
Test: fix occasional 069.memory_leak_extended test failure.
It turned out that reason of the occasional test failure is, pgpool
child process is gone before running ps command after pgbench
finishes. The cause is a kind mismatch FATAL error, "DISCARD ALL
cannot be executed within a pipeline". To fix this, run pgbench in
background and get the process size before pgbench finishes.
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2023-May/004338.html
Bo Peng [Wed, 17 May 2023 12:21:36 +0000 (21:21 +0900)]
Prepare 4.2.13
Bo Peng [Wed, 17 May 2023 12:17:26 +0000 (21:17 +0900)]
Bump version to 4.2.13
Bo Peng [Wed, 17 May 2023 12:11:20 +0000 (21:11 +0900)]
Doc: modify release notes.
Bo Peng [Wed, 17 May 2023 11:48:49 +0000 (20:48 +0900)]
Doc: add release notes.
Tatsuo Ishii [Fri, 12 May 2023 11:41:36 +0000 (20:41 +0900)]
Fix possible dead lock with shared relation cache.
When a user defined function obtains table locking, call to the
function using extended query protocol could lead to dead lock. Here's
a scenario.
(1) In session A client sends parse, bind, execute request for the
function to pgpol.
(2) Pgpool in session A forwards the request to PostgreSQL.
(3) PostgreSQL in session A performs execute, resulting in a table lock.
(4) In session B client sends parse, bind, execute request for the
function to pgpool.
(5) Pgpool in session B forwards the request to PostgreSQL.
(6) PostgreSQL in session B performs bind but the table was already
locked by PostgreSQL in session A, and it waits for release of the
lock.
(7) After pgpool in session B forwards the execute to PostgreSQL, it
acquires semaphore to search shared relation cache to check the
volatility of the function. Then it calls do_query and sends flush
message to PostgreSQL to obtain the response from PostgreSQL up to
this point. But since PostgreSQL is waiting for table lock in #6,
pgpool has to wait for messages beyond bind complete.
(8) After pgpool in session A forwards the execute to PostgreSQL, it
tries to acquire semaphore to search shared relation cache to check
the volatility of the function but the semaphore was already
acquired by pgpool in session B, it waits for the release of the
semaphore.
(9) Session A and session B wait for each other, resulting in a dead
lock.
To fix this, modify pool_search_relcache() so that it releases
semaphore before calling do_query() (but acquires semaphore after
do_query()). By this, session A in #8 above can get semaphore, and go
forward. It receives sync message from client and forwards it to
PostgreSQL. Upon receiving sync, the user defined function will finish
the execution and releases the table lock. This makes PostgreSQL in
session B move forward because now it can obtain the table lock.
Back patch to 4.3, 4.2 and 4.1. On these versions semaphore based
shared release cache is used.
Tatsuo Ishii [Fri, 12 May 2023 08:16:01 +0000 (17:16 +0900)]
Test: stabilize some tests.
069.memory_leak_extended, 070.memory_leak_extended_memqcache and
073.pg_terminate_backend uses "sleep 1" to confirm pgpool starting up,
which makes the tests unstable because there's no guarantee that pgpool
becomes ready within 1 second. Use wait_for_pgpool_startup instead to
stabilize the tests.
Tatsuo Ishii [Sat, 29 Apr 2023 12:44:59 +0000 (21:44 +0900)]
Test: fix regress.sh to show correct number of total tests.
Previously it ignored the number of timed out tests. As a result total
number of tests showed incorrect number because num-total tests was
calculated as num-ok + num-failed.
This is an oversight when timeout was introduced in
6688332da.
Chen Ningwei [Tue, 25 Apr 2023 01:34:26 +0000 (10:34 +0900)]
Doc: add explanation for wd_priority.
Tatsuo Ishii [Thu, 20 Apr 2023 06:53:39 +0000 (15:53 +0900)]
Fix merging mistakes in the previous commit.
Tatsuo Ishii [Wed, 19 Apr 2023 10:41:21 +0000 (19:41 +0900)]
Doc: add restriction regarding PREPARE/EXECUTE/DEALLOCATE.
Bo Peng [Wed, 19 Apr 2023 08:18:01 +0000 (17:18 +0900)]
Update src/Makefile.in.
Tatsuo Ishii [Wed, 19 Apr 2023 07:56:24 +0000 (16:56 +0900)]
Add src/utils/psqlscan.c.
V4.4 or older branches still keep generated files in git repository. I
forgot this in commit:
https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=
77f1e64d715c25d24830623f705877030d34e768
Tatsuo Ishii [Wed, 19 Apr 2023 03:40:48 +0000 (12:40 +0900)]
Doc: add caution to use -D option with pgpool.
Tatsuo Ishii [Sun, 9 Apr 2023 04:11:12 +0000 (13:11 +0900)]
Doc: remove unnecessary restriction regarding multi-statement.
This applies to master branch only for now because this needs
commit:
https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=
48da8715bf403965507eef0321c0ab10054ac71c
If we decide to back port this, we will apply this change to stable
branches.
Tatsuo Ishii [Sat, 25 Mar 2023 07:21:27 +0000 (16:21 +0900)]
Judge multi statement query using psqlscan.
Psqlscan is a module in the PostgreSQL source tree. It is essentially
subset of PostgreSQL SQL scanner but it is specialized for detecting
the end of each SQL statement. Therefore we can count the number of
SQL statements in a query string by using it.
Note that we use psqlscan only when query string is large.
Because psqlscan is not designed as an external library, it is hard to
call it from outside of PostgreSQL source tree. So I decided to import
psqlscan source code. This module consists of multiple files. Program
sources are deployed in src/utils directory and header files are
deployed in src/include/utils directory.
psqlscan module was originally designed for frontend programs and uses
malloc directly. So I changed them so that it calls palloc and
friends. Additionally pgstrcasecmp.c and pqexpbuffer.c are also
imported. They are used by psqlscan.
The example usage of psqlscan can be found in
multi_statement_query():src/protocol/pool_proto_modules.c.
Discussion:
https://www.pgpool.net/pipermail/pgpool-hackers/2023-February/004291.html
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2023-April/004320.html
Tatsuo Ishii [Sat, 15 Apr 2023 07:26:23 +0000 (16:26 +0900)]
Test: skip the test if there's no test.sh.
This is useful when developers want to run the test on git repository
because the check suppresses false positive errors. There could newer
branch's test directories remain without any contents and regress.sh
reports wrong failures.
Bo Peng [Fri, 14 Apr 2023 05:26:48 +0000 (14:26 +0900)]
Doc: fix PostgreSQL version of the previous commit.
Bo Peng [Fri, 14 Apr 2023 04:00:59 +0000 (13:00 +0900)]
Doc: Enhancing installation documentation.
- add detailed decriptions of packages
- mention that pgpool-II-pgXX-extensions needs to be installed on PostgreSQL servers
Tatsuo Ishii [Thu, 13 Apr 2023 00:25:09 +0000 (09:25 +0900)]
Doc: add "Backing up PostgreSQL database" section to "Server Setup and Operation" chapter.
Bo Peng [Thu, 13 Apr 2023 09:00:08 +0000 (18:00 +0900)]
Doc: remove the configuration of "- D" start OPTS from "8.2. Pgpool-II + Watchdog Setup Example".
Tatsuo Ishii [Thu, 6 Apr 2023 03:43:06 +0000 (12:43 +0900)]
Fix occasional 005.jdbc test failure.
The direct cause of the error is:
2023-02-22 08:51:47.705: PostgreSQL JDBC Driver pid 12420: LOG: Parse: Error or notice message from backend: : DB node id: 0 backend pid: 12488 statement: "COMMIT" message: "prepared statement "S_1" already exists"
Actually the root of the error is this:
2023-02-22 08:51:45.242: PostgreSQL JDBC Driver pid 12420: LOG: pool_send_and_wait: Error or notice message from backend: : DB node id: 0 backend pid: 12488 statement: "DISCARD ALL" message: "DISCARD ALL cannot be executed within a pipeline"
"DISCARD ALL" was generated by pgpool (reset_query_list) to discard
some objects including prepared statements created in the
session. Since DISCARD ALL failed, the prepared statement S_1 was not
removed. Thus the next session failed because S_1 already existed.
To fix this, new global boolean flag reset_query_error is
introduced. The flag is set inside pool_send_and_wait() when a reset
query executed by SimpleQuery() results in ERROR. If the flag is true,
backend_cleanup() discards the backend connection so that any objects,
including named statement, corresponding to the session is discarded
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2023-February/004293.html
Tatsuo Ishii [Sun, 9 Apr 2023 02:36:30 +0000 (11:36 +0900)]
Doc: fix restrictions section.
- Add mention about SCRAM-SHA-256 authentication
- Add mention about snapshot isolation mode
Tatsuo Ishii [Mon, 3 Apr 2023 23:21:28 +0000 (08:21 +0900)]
Doc: add index for online recovery.
Tatsuo Ishii [Thu, 30 Mar 2023 00:15:42 +0000 (09:15 +0900)]
More schema qualification fix.
This is a follow up to:
https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=
49ca4800d1a804c1f6ef5807d9fbeeef85888fb6
Back patch to all supported branches: 4.4 to 4.0
Tatsuo Ishii [Tue, 28 Mar 2023 01:55:59 +0000 (10:55 +0900)]
Use schema qualification for internal queries.
Some of objects such as function and cast did not use "pg_catalog."
schema qualification. This does not lead to immediate security
concern but using the schema qualification is always good practice.
Not that for this I had to increase some buffer length:
- POOL_RELCACHE.query was changed from 1024 to 1500.
- query buffer size in pool_search_relcache was changed from 1024 to 1500.
Back patch to all supported branches: 4.4 to 4.0
Tatsuo Ishii [Tue, 28 Mar 2023 00:49:08 +0000 (09:49 +0900)]
Doc: fix typo in the description of backend_application_name.
Tatsuo Ishii [Wed, 22 Mar 2023 11:53:38 +0000 (20:53 +0900)]
Fix compile error on systems using musl libc.
Patch provided by leimaohui.
https://www.pgpool.net/mantisbt/view.php?id=790
Tatsuo Ishii [Wed, 22 Feb 2023 01:20:15 +0000 (10:20 +0900)]
Doc: mention that AES256 support requires --with-openssl option.