pgpool2.git
2 years agoFix SPEC file typo.
Bo Peng [Sun, 25 Dec 2022 11:33:24 +0000 (20:33 +0900)]
Fix SPEC file typo.

2 years agoFix compiler warning.
Tatsuo Ishii [Sat, 24 Dec 2022 03:41:01 +0000 (12:41 +0900)]
Fix compiler warning.

pgpool-regclass.c needed to include "utils/varlena.h".  Also sort out
the header files order (except postgres.h which needs to be appear at
the top among PostgreSQL header files).

The compiler warning was reported by Florian Weimer.
https://www.pgpool.net/pipermail/pgpool-hackers/2022-December/004241.html

2 years agoFix SPEC file typo.
Bo Peng [Fri, 23 Dec 2022 14:16:43 +0000 (23:16 +0900)]
Fix SPEC file typo.

2 years agoFix SPEC file typo.
Bo Peng [Fri, 23 Dec 2022 07:51:01 +0000 (16:51 +0900)]
Fix SPEC file typo.

2 years agoUpdate SPEC file to change /lib/tmpfiles.d/ file from /var/run to /run.
Bo Peng [Fri, 23 Dec 2022 07:28:52 +0000 (16:28 +0900)]
Update SPEC file to change /lib/tmpfiles.d/ file from /var/run to /run.

2 years agoPrepare 4.2.11 V4_2_11 V4_2_11_RPM
Bo Peng [Thu, 22 Dec 2022 06:46:33 +0000 (15:46 +0900)]
Prepare 4.2.11

2 years agoDoc: Bump doc version to 4.2.11
Bo Peng [Thu, 22 Dec 2022 06:42:05 +0000 (15:42 +0900)]
Doc: Bump doc version to 4.2.11

2 years agoDoc: update repo package version.
Bo Peng [Thu, 22 Dec 2022 06:28:38 +0000 (15:28 +0900)]
Doc: update repo package version.

2 years agoDoc: add release note.
Bo Peng [Thu, 22 Dec 2022 06:11:16 +0000 (15:11 +0900)]
Doc: add release note.

2 years agoFix for :[pgpool-hackers: 4227] Issue with failover_require_consensus
Muhammad Usama [Mon, 19 Dec 2022 19:39:32 +0000 (00:39 +0500)]
Fix for :[pgpool-hackers: 4227] Issue with failover_require_consensus

The fix is to dynamically set the failover command timeout based on the maximum
value of health check parameters across the watchdog cluster.

Reviewed and tested by Tatsuo Ishii

2 years agoDoc: fix Japanese documentation to sync with English docs.
Tatsuo Ishii [Mon, 19 Dec 2022 04:50:33 +0000 (13:50 +0900)]
Doc: fix Japanese documentation to sync with English docs.

2 years agoAllow to define PGPOOLDIR uing environment variable.
Bo Peng [Sat, 17 Dec 2022 00:57:37 +0000 (09:57 +0900)]
Allow to define PGPOOLDIR uing environment variable.

per bug766.

2 years agoDoc: fix typo in Aurora example.
Tatsuo Ishii [Wed, 14 Dec 2022 08:38:29 +0000 (17:38 +0900)]
Doc: fix typo in Aurora example.

2 years agoFix typo in comments.
Tatsuo Ishii [Sat, 10 Dec 2022 06:23:30 +0000 (15:23 +0900)]
Fix typo in comments.

2 years agoDowngrade LOG messages "new IPC connection received" to DEBUG1.
Bo Peng [Sat, 3 Dec 2022 13:31:15 +0000 (22:31 +0900)]
Downgrade LOG messages "new IPC connection received" to DEBUG1.

It is a normal messages and should not be logged as LOG.
Patch is created by pstef and reviewed by Bo Peng.

2 years agoFix not to print warnings of use of backslashes in parser.
Tatsuo Ishii [Thu, 24 Nov 2022 06:31:48 +0000 (15:31 +0900)]
Fix not to print warnings of use of backslashes in parser.

When standard_conforming_strings = off and escape_string_warning = on,
PostgreSQL prints warnings if backslashes are used in string
literal. This is fine. But previously Pgpool-II's parser printed the
same message too. This is redundant.

2 years agoImprove follow_primary.sh.sample script:
Bo Peng [Mon, 7 Nov 2022 04:49:06 +0000 (13:49 +0900)]
Improve follow_primary.sh.sample script:
- run checkpoint command on primary to update control file before running pg_rewind
- check if a directory exists before removing it

2 years agoDeal with idle_session_timeout.
Tatsuo Ishii [Wed, 26 Oct 2022 23:45:17 +0000 (08:45 +0900)]
Deal with idle_session_timeout.

If idle_session_timeout (added in PostgreSQL 14) is enabled and the
timeout fires, followings happen:

- If failover_on_backend_error is on (the default), Pgpool-II will
  trigger failover.

- If only one of PostgreSQL servers enables idle_session_timeout,
  Pgpool-II could hang.

To deal with idle_session_timeout detect_idle_session_timeout_error()
is added to detect the error code for idle_session_timeout. If the
error is detected, Pgpool-II returns the error code to frondend as a
fatal error and disconnects the session.  This is a similar fix
implemented for idle_in_transaction_session_timeout.

https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=3f5986eee360f12e6a0bb77aa46f95abf5f6bc10

Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2022-October/004209.html
Back-path-through: 4.0

2 years agoDoc: enhance description about memqcache_method.
Tatsuo Ishii [Fri, 7 Oct 2022 02:26:28 +0000 (11:26 +0900)]
Doc: enhance description about memqcache_method.

Add explanation which method should be used.

Backpatch-through: 3.7.

2 years agoDeal with SSL error SSL_ERROR_ZERO_RETURN.
Tatsuo Ishii [Wed, 14 Sep 2022 02:16:03 +0000 (11:16 +0900)]
Deal with SSL error SSL_ERROR_ZERO_RETURN.

Previously this caused failover, which was actually unnecessary because
it means the server is just going to close the connection.

Discussion: https://www.pgpool.net/pipermail/pgpool-general/2022-September/008425.html
Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2022-September/004194.html

2 years agoFix comment in is_select_query().
Tatsuo Ishii [Mon, 3 Oct 2022 10:21:40 +0000 (19:21 +0900)]
Fix comment in is_select_query().

This function returns false for SELECT INTO, rather than plain SELECT.

2 years agoFix: Setting memory cache size greater than 2GB causes a segfault.
Muhammad Usama [Fri, 30 Sep 2022 20:33:12 +0000 (01:33 +0500)]
Fix: Setting memory cache size greater than 2GB causes a segfault.

The problem was in the block_address() function that returns the memory address
for a given cache block, It was using 32bit integers to calculate the offset of
the block within the shared memory space that is only good until the 2GB limit.

2 years agoFix rare segfaults in pcp_proc_info, SHOW pool_pools and SHOW pool_processes.
Tatsuo Ishii [Fri, 23 Sep 2022 06:46:33 +0000 (15:46 +0900)]
Fix rare segfaults in pcp_proc_info, SHOW pool_pools and SHOW pool_processes.

The segfaults were in get_pools() and get_processes(). They first
extracted pid of particular process info slot on shared memory then
searched the slot again by using pid as the key. Because these steps
were not protected by any locking, it was possible that the search
using the pid failed and returned NULL if the process id is
overwritten by pgpool parent which is responsible for forking new
child process after the process exiting. As a result any subsequent
reference to the NULL pointer generated segfaults.

Solution is, first get the pointer to the process info slot then
extract the process id member from the pointer. This way, still
concurrent updating to the shared memory info by the parent process is
possible (which may lead to strange results in the output) but at
least we can avoid segfaults.

3 years agoFix to not allow Unix-domain socket path with invalid length.
Masaya Kawamoto [Mon, 12 Sep 2022 09:18:21 +0000 (09:18 +0000)]
Fix to not allow Unix-domain socket path with invalid length.

3 years agoDoc: mention that health check process may use SSL.
Tatsuo Ishii [Sun, 28 Aug 2022 03:54:51 +0000 (12:54 +0900)]
Doc: mention that health check process may use SSL.

Also mention that streaming replication check may use SSL too.
This should have been added since 2010.

Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2022-August/004188.html

3 years agoDoc: fix typo.
Bo Peng [Thu, 18 Aug 2022 05:18:20 +0000 (14:18 +0900)]
Doc: fix typo.

3 years agoPrepare 4.2.10 V4_2_10 V4_2_10_RPM
Bo Peng [Wed, 17 Aug 2022 08:30:52 +0000 (17:30 +0900)]
Prepare 4.2.10

3 years agoDoc: update doc version.
Bo Peng [Wed, 17 Aug 2022 08:23:58 +0000 (17:23 +0900)]
Doc: update doc version.

3 years agoSetting PGPOOLDIR before using it set SAMPLE_CONF when TEST_SAMPLES is enabled.
Bo Peng [Wed, 17 Aug 2022 07:29:15 +0000 (16:29 +0900)]
Setting PGPOOLDIR before using it set SAMPLE_CONF when TEST_SAMPLES is enabled.

3 years agoDoc: add release notes.
Bo Peng [Tue, 16 Aug 2022 05:45:17 +0000 (14:45 +0900)]
Doc: add release notes.

3 years agoDoc: enhance description in the memory requirement section.
Tatsuo Ishii [Tue, 26 Jul 2022 06:23:40 +0000 (15:23 +0900)]
Doc: enhance description in the memory requirement section.

Add explanation about memory usage while pgpool child process is running.

3 years agoDoc: fix typo.
Tatsuo Ishii [Tue, 26 Jul 2022 02:31:44 +0000 (11:31 +0900)]
Doc: fix typo.

3 years agoDoc: mention that certain SELECTs are not cached.
Tatsuo Ishii [Tue, 19 Jul 2022 13:14:41 +0000 (22:14 +0900)]
Doc: mention that certain SELECTs are not cached.

SELECTs including TIMESTAMP WITH TIMEZONE or TIME WITH TIMEZONE
SELECTs including CAST to TIMESTAMP WITH TIMEZONE or TIME WITH TIMEZONE
SELECTs including SQLValueFunction (CURRENT_TIME, CURRENT_USER etc.)

3 years agoTest: print Pgpool-II version in the regression test.
Tatsuo Ishii [Sun, 10 Jul 2022 07:04:49 +0000 (16:04 +0900)]
Test: print Pgpool-II version in the regression test.

3 years agoTest: Fix regression test script to look for pgpool.conf in the proper install directory.
Tatsuo Ishii [Sat, 9 Jul 2022 09:41:18 +0000 (18:41 +0900)]
Test: Fix regression test script to look for pgpool.conf in the proper install directory.

regress.sh did set PGPOOLDIR environment variable despite that
pgpool_setup look for pgpool.conf sample files at $PGPOOLDIR/etc. As a
result, pgpool_setup looked for pgpool.conf.sample at default install
directory (usually /usr/local/etc) even if "-m noinstall" is not
set. If inappropriate pgpool.conf sample files are in
/usr/local/etc., regression test fails.

3 years agoFix to not cache SELECT having functions with return types are timestamptz or timetz.
Tatsuo Ishii [Wed, 6 Jul 2022 12:31:08 +0000 (21:31 +0900)]
Fix to not cache SELECT having functions with return types are timestamptz or timetz.

Functions with return type is timestamptz or timetz is affected by time zone setting.
Consider following scenario:

1) SELECT having such functions gets called and cache created.

2) time zone is changed.

3) same SELECT is called and the cache is used. The cache value is
correct any more because of the time zone change.

Discussion: https://www.pgpool.net/pipermail/pgpool-general/2022-July/008367.html

3 years agoFix query cache to not cache timestamptz and timetz cast.
Tatsuo Ishii [Tue, 5 Jul 2022 05:06:10 +0000 (14:06 +0900)]
Fix query cache to not cache timestamptz and timetz cast.

Even if query is constant + cast (like '2022-07-05
14:07:00'::timestamptz), the result can be changed by SET TIME ZONE
command etc. and the result should not be cached.
Also regression test data added.

Discussion: https://www.pgpool.net/pipermail/pgpool-general/2022-July/008353.html

3 years agoFix bug in query cache.
Tatsuo Ishii [Mon, 4 Jul 2022 05:23:48 +0000 (14:23 +0900)]
Fix bug in query cache.

[pgpool-general: 8285] Timestamp cast not cached

reported that query like "Select '2022-02-18
07:00:00.006547’::timestamp" is not cached.

The function non_immutable_function_call_walker() which judges whether
the query contains non immutable functions mistakenly assumes that any
query including cast to timestamp etc. should not to be cached. These
codes were originally added to detect CURRENT_TIMESTAMP etc. as they
are transformed to type cast in raw parser. Unfortunately this is
overkill since "'2022-02-18 07:00:00.006547’::timestamp" is also
transformed to a type cast.

Fortunately Pgpool-II 3.7 and after imported PostgreSQL 10 or newer
parser, which transforms CURRENT_TIMESTAMP etc. to SQLValueFunction.

As a result, the type cast handling code in
non_immutable_function_call_walker() is not necessary anymore. So this
commit removed the code.

Note: an interested thing in the report is, "Select '2022-02-18
07:00:00.006547’::timestamp" is not cached while "Select '2022-02-18
07:00:00.006547’::date" is cached. Why? Well
non_immutable_function_call_walker() (wrongly) assumes that the type
cache is always created by SystemTypeName (a parser's function), which
always adds pg_catalog schema. This only happens when the type name is
a reserved keyword. TIMESTAMP is a reserved keyword (and TIME too). So
non_immutable_function_call_walker() catches TIMESTAMP cast as
expected, but DATE is not a reserved keyword and it is transformed to
type name without "pg_catalog" schema. So
non_immutable_function_call_walker() misses it, and it is cached.

3 years agoAdd ssh options to restore_command in sample scripts.
Bo Peng [Sun, 3 Jul 2022 12:09:17 +0000 (21:09 +0900)]
Add ssh options to restore_command in sample scripts.

Patch is created by Jon SCHEWE and updated by Bo Peng.

3 years agoDoc: fix description about using PCP password file when connect to Unix domain socket
Masaya Kawamoto [Thu, 30 Jun 2022 03:58:37 +0000 (03:58 +0000)]
Doc: fix description about using PCP password file when connect to Unix domain socket

The localhost entry in pcppass matches only for the default PCP socket
directory path, not all Unix socket connections. This behavior is the
same as pgpass.

3 years agoAllow to run pgpool_setup on PostgreSQL 15.
Tatsuo Ishii [Sun, 26 Jun 2022 09:11:08 +0000 (18:11 +0900)]
Allow to run pgpool_setup on PostgreSQL 15.

basebackup.sh created by pgpool_setup uses pg_start_backup() and
pg_stop_backup().  PostgreSQL 15 renamed them. But there's more
problem. The renamed functions (pg_backup_start() and
pg_backup_stop()) must be called within a same connection. Adapting
these requires non-trivial changes. So I decided to rewrite the code
to not use pg_start_backup() and pg_stop_backup() so that it uses
pg_basebackup command instead.

Per https://www.pgpool.net/mantisbt/view.php?id=757

3 years agoAllow pgpool_setup to test sample scripts and sample config files contained in RPMs.
Bo Peng [Sun, 19 Jun 2022 09:51:38 +0000 (18:51 +0900)]
Allow pgpool_setup to test sample scripts and sample config files contained in RPMs.

Currently, pgpool_setup generates scripts (e.g. failover.sh, follow_primary.sh) to test pgpool.

This commit addes a new option "-c" in src/test/regression/regress.sh
and a new environment variables TEST_SAMPLES in pgpool_setup.
Option "-c" enables TEST_SAMPLES and allows pgpool_setup to test sample scripts and
sample config files contained in RPMs.

This commit also changes "backend_hostnameX = '/tmp'" to "backend_hostnameX = 'localhost'".

3 years agoAdd pool_config.c to src/tools/pgmd5/.gitignore.
Tatsuo Ishii [Mon, 13 Jun 2022 07:09:46 +0000 (16:09 +0900)]
Add pool_config.c to src/tools/pgmd5/.gitignore.

This should avoid the error:
error: The following untracked working tree files would be overwritten by checkout:
src/tools/pgmd5/pool_config.c

3 years agoDoc: fix memory requirement section.
Tatsuo Ishii [Mon, 6 Jun 2022 13:06:37 +0000 (22:06 +0900)]
Doc: fix memory requirement section.

- enhance the formula to calculate shared memory requirement so that it computes more accurate result.

- fix shared memory requirement for shared rel cache. The old value 64MB was simply wrong.

- fix process memory requirement. Previously the formula was based on
  RSS. However PSS should be used because RSS includes shared memory
  such as the memory used for libraries. This resuls in lot smaller
  memory requirement than before.

3 years agoAllow to rewrite archive_command in sample scripts.
Bo Peng [Sun, 5 Jun 2022 13:51:52 +0000 (22:51 +0900)]
Allow to rewrite archive_command in sample scripts.

3 years agoUpdate PSQL connection information in sample scripts.
Bo Peng [Sun, 5 Jun 2022 06:44:05 +0000 (15:44 +0900)]
Update PSQL connection information in sample scripts.

3 years agoUpdated the sample scripts to allow ssh login user and ssh key file to be set using...
Bo Peng [Sat, 4 Jun 2022 16:21:35 +0000 (01:21 +0900)]
Updated the sample scripts to allow ssh login user and ssh key file to be set using variables.

3 years agoWhen CloseComplete is received, foward to frontend without buffering.
Tatsuo Ishii [Tue, 31 May 2022 12:23:48 +0000 (21:23 +0900)]
When CloseComplete is received, foward to frontend without buffering.

It seems occasional timeour error in 074.bug700_memqcache_segfault seems to be caused by the buildfarm log as of
[pgpool-buildfarm: 2158] Pgpool-II buildfarm results CentOS7
* master  PostgreSQL 12  CentOS7
testing 074.bug700_memqcache_segfault...timeout.

From the pgpool.log:
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  pool_discard_temp_query_cache: cache discarded: 0x1be53e8
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  memcache reset buffer
2022-05-28 17:14:52.612: pgproto pid 45886: DETAIL:  discard: 0x1be57a8
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  memcache discarding query cache array
2022-05-28 17:14:52.612: pgproto pid 45886: DETAIL:  num_caches: 0
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  memcache reset buffer
2022-05-28 17:14:52.612: pgproto pid 45886: DETAIL:  create: 0x1be57a8
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  memcache reset buffer
2022-05-28 17:14:52.612: pgproto pid 45886: DETAIL:  discard temp buffer of 0x1be7c08 (SELECT 1)
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  CloseComplete: remove sent message. kind:B, name:P1
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  processing command complete
2022-05-28 17:14:52.612: pgproto pid 45886: DETAIL:  set transaction state to T
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  memcache reset buffer
2022-05-28 17:14:52.612: pgproto pid 45886: DETAIL:  discard: 0x1be57a8
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  memcache discarding query cache array
2022-05-28 17:14:52.612: pgproto pid 45886: DETAIL:  num_caches: 0
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  memcache reset buffer
2022-05-28 17:14:52.612: pgproto pid 45886: DETAIL:  create: 0x1be57a8
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  memcache reset buffer
2022-05-28 17:14:52.612: pgproto pid 45886: DETAIL:  discard temp buffer of 0x1be7c08 (SELECT 1)
2022-05-28 17:14:52.612: pgproto pid 45886: DEBUG:  CloseComplete: remove sent message. kind:B, name:P2

It seems the CloseComplete message was received from backend but if looking into results.txt:

FE=> Query (query="SET statement_timeout TO 1000")
<= BE CommandComplete(SET)
<= BE ReadyForQuery(I)
FE=> Parse(stmt="S1", query="SELECT 1")
FE=> Bind(stmt="S1", portal="P1")
FE=> Execute(portal="P1")
FE=> Close(portal="P1")
FE=> Bind(stmt="S1", portal="P2")
FE=> Execute(portal="P2")
FE=> Close(portal="P2")
FE=> Sync
<= BE ParseComplete
<= BE BindComplete
<= BE DataRow
<= BE CommandComplete(SELECT 1)
<= BE CloseComplete
<= BE BindComplete
<= BE DataRow
<= BE CommandComplete(SELECT 1)

It seems the frontend did not receive the CloseComplete
message. SimpleForwardToFrontend() did buffering when it forwarded the
CloseComplete message. I guess this is the cause of the timeout. So
let's change SimpleForwardToFrontend() so that it forwards
CloseComplete message without buffering and see what buildfarm says.

3 years agoFix segfault in end_internal_transaction().
Tatsuo Ishii [Thu, 2 Jun 2022 07:55:10 +0000 (16:55 +0900)]
Fix segfault in end_internal_transaction().

This was introduced by the commit:
https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=050be475d738ab4f8268dce21e2e5361b7dbcbee

In raw mode, end_internal_transaction() should not be called.

3 years agoDoc: fix wrong explanation on memqcache_maxcache, memqcache_expire.
Tatsuo Ishii [Thu, 2 Jun 2022 01:03:12 +0000 (10:03 +0900)]
Doc: fix wrong explanation on memqcache_maxcache, memqcache_expire.

Those parameters cannot be changed by reloading config file. Restarting
pgpool is required.

Discussion: https://www.pgpool.net/pipermail/pgpool-general/2022-June/008254.html

3 years agoFix internal transaction handling bug in snapshot isolation mode.
Tatsuo Ishii [Tue, 31 May 2022 10:46:16 +0000 (19:46 +0900)]
Fix internal transaction handling bug in snapshot isolation mode.

When SELECT is executed in snapshot isolation mode, it is surrounded
in an "internal transaction".  For example if there are 3 backends,
each backend starts an internal transaction with "BEGIN" command.
However, there was an oversight in end_internal_transaction(), which
is responsible for closing the transaction. It only closed the
transaction on "load balance node". This causes a problem with certain
query following SELECT, for example VACUUM. Suppose the SELECT is
executed on backend node 2, then transaction on node 0 and node 1 were
not closed. If VACUUM is executed on node 0, it fails because the
transaction is not closed on node 0 (remember, VACUUM cannot be
executed in an explicit transaction).

Also add a test case for this to the 030.snapshot_isolation test.

3 years agoFix bug in src/redhat/pgpool_log.patch.
Bo Peng [Tue, 31 May 2022 06:22:55 +0000 (15:22 +0900)]
Fix bug in src/redhat/pgpool_log.patch.

3 years agoAdd volatile modifier to a variable used in the query cache module.
Tatsuo Ishii [Fri, 20 May 2022 04:55:20 +0000 (13:55 +0900)]
Add volatile modifier to a variable used in the query cache module.

"sts" variable used in pool_fetch_memory_cache() did not have volatile
modifier although it is used in PG_TRY() block. If an exception arises
the value set to variable will be lost in the PG_CATCH block. The bad
effect of this will not be triggered unless error occurs and I think
the bad effect has rarely been observed in the wild (as for as I know,
I have never heard such a report). Anyway, bug is bug.

3 years agoUpdate patch file. V4_2_9 V4_2_9_RPM
Bo Peng [Wed, 18 May 2022 07:13:02 +0000 (16:13 +0900)]
Update patch file.

3 years agoUpdate patch file.
Bo Peng [Wed, 18 May 2022 06:33:55 +0000 (15:33 +0900)]
Update patch file.

3 years agoPrepare 4.2.9
Bo Peng [Wed, 18 May 2022 02:42:21 +0000 (11:42 +0900)]
Prepare 4.2.9

3 years agoDoc: update doc version.
Bo Peng [Wed, 18 May 2022 02:33:42 +0000 (11:33 +0900)]
Doc: update doc version.

3 years agoDoc: add release notes.
Bo Peng [Wed, 18 May 2022 02:13:54 +0000 (11:13 +0900)]
Doc: add release notes.

3 years agoDoc: move the example of "Pgpool-II on Kubernetes" to https://github.com/pgpool/pgpoo...
Bo Peng [Wed, 18 May 2022 00:14:28 +0000 (09:14 +0900)]
Doc: move the example of "Pgpool-II on Kubernetes" to https://github.com/pgpool/pgpool2_on_k8s/tree/master/docs.

3 years agoEnhance in stopping pgpool main process.
Tatsuo Ishii [Sat, 14 May 2022 00:44:21 +0000 (09:44 +0900)]
Enhance in stopping pgpool main process.

If "pgpool stop" couldn't terminate the main process within certain
period (currently 5 seconds), send the signal again. The reason why
pgpool does not accept the stopping signal is not clear at the moment,
I expect this reduces errors of buildfarm timeout. See if this works.

3 years agoFix comment.
Tatsuo Ishii [Tue, 10 May 2022 05:20:26 +0000 (14:20 +0900)]
Fix comment.

3 years agoDoc: update configuration example.
Masaya Kawamoto [Mon, 9 May 2022 05:23:46 +0000 (05:23 +0000)]
Doc: update configuration example.

3 years agoChange the PID length of pcp_proc_count command to 7 characters long.
Bo Peng [Mon, 2 May 2022 05:15:32 +0000 (14:15 +0900)]
Change the PID length of pcp_proc_count command to 7 characters long.

3 years agoFix for [pgpool-general: 7896] Possible race condition..
Muhammad Usama [Wed, 27 Apr 2022 10:30:58 +0000 (15:30 +0500)]
Fix for [pgpool-general: 7896] Possible race condition..

Watchdog does not allow the remote nodes reported lost by life-check to rejoin
the cluster until the life-check process confirms the existence of life in the
previously lost nodes. This is good enough except for the case when the
(lost by life-check) node tries to rejoin the cluster after it was restarted
(Pgpool-II service restarted).
What happens is the cluster keeps rejecting the restarted node because
the cluster's life-check doesn't agree while the restarted node's life-check
waits to be added to cluster before it can start sending the heart-beats.

The fix is to allow the previously lost remote node become the part of the
cluster after restart, no matter the lost-reason.

Issue report:
https://www.pgpool.net/pipermail/pgpool-general/2021-November/007954.html

3 years agoDoc: update the example output of pcp_watchdog_info.
Bo Peng [Thu, 28 Apr 2022 09:03:10 +0000 (18:03 +0900)]
Doc: update the example output of pcp_watchdog_info.

3 years agoDoc: mention that escaping is required if you are providing the password as an argume...
Bo Peng [Thu, 28 Apr 2022 04:22:52 +0000 (13:22 +0900)]
Doc: mention that escaping is required if you are providing the password as an argument to pg_enc and the password contains a "$" character.

3 years agoTest: stabilize 018.detach_primary.
Tatsuo Ishii [Sun, 17 Apr 2022 12:48:51 +0000 (21:48 +0900)]
Test: stabilize 018.detach_primary.

The test occasionally failed with timeout. The cause seems the test
proceeds before watchdog are ready.  So wait till watchdog is ready
using pcp_watchdog_info.

3 years agoImprove regression test to detect segmentation fault.
Bo Peng [Mon, 18 Apr 2022 15:36:54 +0000 (00:36 +0900)]
Improve regression test to detect segmentation fault.

3 years agoTest: enhance 074.bug700_memqcache_segfault
Tatsuo Ishii [Sun, 17 Apr 2022 04:52:14 +0000 (13:52 +0900)]
Test: enhance 074.bug700_memqcache_segfault

Set statement_timeout at the beginning of the test to avoid the time
out error because it takes long time before failure.

3 years agoFix exit_handler in pgpool main process.
Tatsuo Ishii [Thu, 14 Apr 2022 11:01:56 +0000 (20:01 +0900)]
Fix exit_handler in pgpool main process.

It was allowed to be interrupted by signals (SIGTERM, SIGINT and
SOGQUIT) while the exit_handler. As a resit, exit_handler is executed
while executing exit_handler. This could cause infinite wait in
terminate_childrens() which is called from exit_handler.  To prevent
this, protect variable "exiting" using semaphore to make sure that
only one instance of exit_handler runs at the same time.

Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2022-April/004149.html

3 years agoFix memory leak pointed out by Coverity.
Masaya Kawamoto [Mon, 11 Apr 2022 07:30:54 +0000 (07:30 +0000)]
Fix memory leak pointed out by Coverity.

3 years agoFix shared memory allocation function.
Tatsuo Ishii [Wed, 6 Apr 2022 07:30:35 +0000 (16:30 +0900)]
Fix shared memory allocation function.

pool_shared_memory_segment_get_chunk() which is responsible for shared
memory allocation, failed to consider request size alignment. If
requeste size is not in MAXALIGN (typically 8) bytes, it could overrun
the shared memory area. Probably harmless in the wild but better to
fix.

3 years agoFix possible null pointer dereference per Coverity.
Tatsuo Ishii [Wed, 6 Apr 2022 05:50:56 +0000 (14:50 +0900)]
Fix possible null pointer dereference per Coverity.

3 years agoRevert "Prevent hang in terminate_all_childrens()."
Tatsuo Ishii [Mon, 4 Apr 2022 07:24:58 +0000 (16:24 +0900)]
Revert "Prevent hang in terminate_all_childrens()."

This reverts commit 33e0ff2d1628d5a382ffbc4378074b4d5a5c5ac4.

3 years agoPrevent hang in terminate_all_childrens().
Tatsuo Ishii [Fri, 1 Apr 2022 11:02:54 +0000 (20:02 +0900)]
Prevent hang in terminate_all_childrens().

waitpid() was used in the function without WNOHANG was being set.
This could cause hang in waitpic().
Also fix typo. Rename terminate_all_childrens to terminate_all_children.

3 years agoFix bug with pg_enc and pg_md5.
Tatsuo Ishii [Fri, 18 Mar 2022 11:52:07 +0000 (20:52 +0900)]
Fix bug with pg_enc and pg_md5.

When these commands are invoked with "-i" option (read username/password
pairs from a file), it did not create proper entries in pool_passwd.
This bug was introduced by the commit:
https://git.postgresql.org/gitweb/?p=pgpool2.git;a=commit;h=441bde41767fe3bccad513735f946dd2dec5059b

Bug reported in https://www.pgpool.net/mantisbt/view.php?id=747

3 years agoEnhance error message while processing parse message.
Tatsuo Ishii [Sat, 12 Mar 2022 04:33:42 +0000 (13:33 +0900)]
Enhance error message while processing parse message.

In non-streaming replication mode, a sync message is sent to backend
after a parse message is sent to backend, expecting to get a ready for
query message.  If different message is returned, pgpool will complain
that. This commit adds more information: the message kind returned and
the backend node id.

3 years agoDowngrade log level of ParameterStatus message.
Tatsuo Ishii [Fri, 11 Mar 2022 01:38:08 +0000 (10:38 +0900)]
Downgrade log level of ParameterStatus message.

In commit 4bcba5258130c3cd9f855157a4359aad2fa7acfc the log level when
ParameterStatus message arrives from backend was changed from DEBUG5
to LOG. There are multiple complains about the change because chances
of the event is more frequent than I thought. So revert back the log
level to DEBUG5.

Discussion: https://www.pgpool.net/pipermail/pgpool-general/2022-March/008101.html

3 years agoFix main process exiting while performing finding primary node.
Tatsuo Ishii [Fri, 4 Mar 2022 11:00:41 +0000 (20:00 +0900)]
Fix main process exiting while performing finding primary node.

Pgpool-II main process tries to find primary node whenever the cluster
status is changed by failover/failback. While doing it, if a backend
is failing or shutting down, socket write to the backend could
fail. Unfortunately in the case do_query() throws FATAL error, which
makes the Pgpool-II main process die like this.

2022-03-04 18:13:12.711: main pid 795826: WARNING:  write on backend 1 failed with error :"Broken pipe"
2022-03-04 18:13:12.711: main pid 795826: DETAIL:  while trying to write data from offset: 0 wlen: 32
2022-03-04 18:13:12.711: main pid 795826: LOG:  notice_backend_error: called from pgpool main. ignored.
2022-03-04 18:13:12.711: main pid 795826: LOG:  unable to flush data to backend
2022-03-04 18:13:12.711: main pid 795826: DETAIL:  do not failover because I am the main process
2022-03-04 18:13:12.711: main pid 795826: FATAL:  Backend throw an error message
2022-03-04 18:13:12.711: main pid 795826: DETAIL:  Exiting current session because of an error from backend
2022-03-04 18:13:12.711: main pid 795826: HINT:  BACKEND Error: "terminating connection due to administrator command"
2022-03-04 18:13:12.715: main pid 795826: LOG:  shutting down

To prevent it, change ereport(FATAL) to ereport(ERROR) in do_query().

3 years agoFix: [pgpool-general: 8030] ... segfaults on CentOS 8
Muhammad Usama [Thu, 3 Mar 2022 10:03:40 +0000 (15:03 +0500)]
Fix: [pgpool-general: 8030] ... segfaults on CentOS 8

Event names array used by debug messages had a missing
entry for WD_EVENT_I_AM_APPEARING_FOUND

3 years agoChange the default value of pcp_listen_addresses from '*' to 'localhost'.
Tatsuo Ishii [Mon, 28 Feb 2022 07:24:38 +0000 (16:24 +0900)]
Change the default value of pcp_listen_addresses from '*' to 'localhost'.

Sync with compiled default.

3 years agoChange the default value of pcp_listen_addresses from '*' to 'localhost'.
Tatsuo Ishii [Mon, 28 Feb 2022 06:52:11 +0000 (15:52 +0900)]
Change the default value of pcp_listen_addresses from '*' to 'localhost'.

'*' was not very secure. Also the default value of listen_address in
Pgpool-II and PostgreSQL are 'localhost'.

3 years agoDoc: update copyright year.
Bo Peng [Sun, 27 Feb 2022 03:36:16 +0000 (12:36 +0900)]
Doc: update copyright year.

3 years agoAdjusting the field name in pcp_watchdog_info.
Muhammad Usama [Tue, 1 Feb 2022 13:57:13 +0000 (18:57 +0500)]
Adjusting the field name in pcp_watchdog_info.

Details in: https://www.pgpool.net/pipermail/pgpool-hackers/2021-December/004070.html

3 years agoFixed follow_primary.sh.sample script to check the status of PostgreSQL using pg_isready.
Bo Peng [Fri, 25 Feb 2022 05:29:02 +0000 (14:29 +0900)]
Fixed follow_primary.sh.sample script to check the status of PostgreSQL using pg_isready.

3 years agoAdd patch to enable parameters related to logging_collector.
Bo Peng [Tue, 22 Feb 2022 04:38:51 +0000 (13:38 +0900)]
Add patch to enable parameters related to logging_collector.

3 years agoPrepare 4.2.8 V4_2_8 V4_2_8_RPM
Masaya Kawamoto [Thu, 17 Feb 2022 01:39:03 +0000 (01:39 +0000)]
Prepare 4.2.8

3 years agoAdd release notes.
Masaya Kawamoto [Thu, 17 Feb 2022 00:26:47 +0000 (00:26 +0000)]
Add release notes.

3 years agoEnhance parameter status handling.
Tatsuo Ishii [Mon, 31 Jan 2022 08:11:02 +0000 (17:11 +0900)]
Enhance parameter status handling.

When a parameter status message arrives from backend, Pgpool-II
memorized it but did not forward to frontend.  This commit allows
forwarding a parameter status message to frontend.

3 years agoFixed the streaming replication check process not to retry if it cannot connect to...
Bo Peng [Thu, 10 Feb 2022 02:02:38 +0000 (11:02 +0900)]
Fixed the streaming replication check process not to retry if it cannot connect to the backend.

If the backend takes so long to respond, the connection times out,
then the streaming replication check process will continue to retry.
This retry causes a long time failover.

3 years agoAdd validations of wd_lifecheck_password and recovery_password format
Masaya Kawamoto [Thu, 10 Feb 2022 01:50:44 +0000 (01:50 +0000)]
Add validations of wd_lifecheck_password and recovery_password format

This feature was reverted once due to regression
test failure by e5989302d7a1c007d88a7b24bc86877c39c50d62.

3 years agoFix missed static declaration.
Tatsuo Ishii [Wed, 9 Feb 2022 23:51:02 +0000 (08:51 +0900)]
Fix missed static declaration.

The static declaration of fork_follow_child() was missing. Also fix
typo in a comment.

3 years agoFixed mistakes introduced in the previous commit.
Bo Peng [Mon, 7 Feb 2022 03:19:01 +0000 (12:19 +0900)]
Fixed mistakes introduced in the previous commit.

3 years agoFixed sample failover script.
Bo Peng [Mon, 7 Feb 2022 03:06:45 +0000 (12:06 +0900)]
Fixed sample failover script.

This script did not consider the case when the old primary node id is "-1".

3 years agoFix failover() to deal with the case when no former primary node exists.
Tatsuo Ishii [Sun, 6 Feb 2022 08:11:52 +0000 (17:11 +0900)]
Fix failover() to deal with the case when no former primary node exists.

Consider a case when no primary node exists when Pgpool-II starts
up. In this case Req_info->primary_node_id is -1. failover() did not
consider this and skipped to call find_primary_node_repeatedly().
Also follow_master_command was not executed if
Req_info->primary_node_id is -1.

This commit fixes the bug above.

Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2022-February/004114.html

3 years agoFix pgpool_setup in failover scrip creation.
Tatsuo Ishii [Sun, 6 Feb 2022 07:23:46 +0000 (16:23 +0900)]
Fix pgpool_setup in failover scrip creation.

When pgpool_set creates failover.sh, it did not consider the case when
no primary server existed.

3 years agoAdd restriction about set_config.
Tatsuo Ishii [Wed, 2 Feb 2022 06:46:50 +0000 (15:46 +0900)]
Add restriction about set_config.

3 years agoFix memory leak pointed out by Coverity.
Tatsuo Ishii [Wed, 2 Feb 2022 01:19:26 +0000 (10:19 +0900)]
Fix memory leak pointed out by Coverity.

Actually it's a false positive.

3 years agoFix health check process issues pointed out by Coverity.
Tatsuo Ishii [Wed, 2 Feb 2022 01:04:06 +0000 (10:04 +0900)]
Fix health check process issues pointed out by Coverity.

Fix possible NULL terminate missing and memory leak when running in test mode.

3 years agoFix long standing bug with pcp_node_info.
Tatsuo Ishii [Mon, 31 Jan 2022 02:45:33 +0000 (11:45 +0900)]
Fix long standing bug with pcp_node_info.

It appears that occasionally pcp_node_info shows backend_status field
as "quarantine" when it should be "down". While pcp_node_info shows
the status, first it checks the backend_status member in BackendInfo
struct. If it is 3, then checks quarantine member. If it is other than
0, then the backend_status field is shown as "quarantine". So if
garbage remains in quarantine member, it is shown as "quarantine". The
BackendInfo struct is transferred from pcp_worker process to pcp
frontend client. Unfortunately when the quarantine member was added by
commit 54af632c, it was forgotten to modify pcp_worker.c and
pcp_frontend.c so that the "quarantine" member is transferred.

Fix is needed to be back patched to 3.7, when the "quarantine" member
was first added.

Discussion: https://www.pgpool.net/pipermail/pgpool-hackers/2022-January/004110.html