Noah Misch [Sun, 20 Apr 2025 15:28:48 +0000 (08:28 -0700)]
Avoid ERROR at ON COMMIT DELETE ROWS after relhassubclass=f.
Commit
7102070329d8147246d2791321f9915c3b5abf31 fixed a similar bug, but
it missed the case of database-wide ANALYZE ("use_own_xacts" mode).
Commit
a07e03fd8fa7daf4d1356f7cb501ffe784ea6257 changed consequences
from silent discard of a pg_class stats (relpages et al.) update to
ERROR "tuple to be updated was already modified". Losing a relpages
update of an ON COMMIT DELETE ROWS table was negligible, but a
COMMIT-time error isn't negligible. Back-patch to v13 (all supported
versions).
Reported-by: Richard Guo <guofenglinux@gmail.com
Reported-by: Robins Tharakan <tharakan@gmail.com>
Discussion: https://postgr.es/m/CAMbWs4-XwMKMKJ_GT=p3_-_=j9rQSEs1FbDFUnW9zHuKPsPNEQ@mail.gmail.com
Backpatch-through: 13
David Rowley [Sun, 20 Apr 2025 10:12:07 +0000 (22:12 +1200)]
Fix issue with ORDER BY / DISTINCT aggregates and FILTER
1349d2790 added support so that aggregate functions with an ORDER BY or
DISTINCT clause could make use of presorted inputs to avoid an implicit
sort within nodeAgg.c. That commit failed to consider that a FILTER
clause may exist that filters rows before the aggregate function
arguments are evaluated. That can be problematic if an aggregate
argument contains an expression which could error out during evaluation.
It's perfectly valid to want to have a FILTER clause which eliminates
such values, and with the pre-sorted path added in
1349d2790, it was
possible that the planner would produce a plan with a Sort node above
the Aggregate to perform the sort on the aggregate's arguments long before
the Aggregate node would filter out the non-matching values.
Here we fix this by inspecting ORDER BY / DISTINCT aggregate functions
which have a FILTER clause to see if the aggregate's arguments are
anything more complex than a Var or a Const. Evaluating these isn't
going to cause an error. If we find any non-Var, non-Const parameters
then the planner will now opt to perform the sort in the Aggregate node
for these aggregates, i.e. disable the presorted aggregate optimization.
An alternative fix would have been to completely disallow the presorted
optimization for Aggrefs with any FILTER clause, but that wasn't done as
that could cause large performance regressions for queries that see
significant gains from
1349d2790 due to presorted results coming in from
an Index Scan.
Backpatch to 16, where
1349d2790 was introduced
Author: David Rowley <dgrowleyml@gmail.com>
Reported-by: Kaimeh <kkaimeh@gmail.com>
Diagnosed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/CAK-%2BJz9J%3DQ06-M7cDJoPNeYbz5EZDqkjQbJnmRyQyzkbRGsYkA%40mail.gmail.com
Backpatch-through: 16
Michael Paquier [Sat, 19 Apr 2025 23:34:38 +0000 (08:34 +0900)]
psql: Split extended query protocol meta-commands in --help=commands
Compared to v17 with only \bind able to do extended query protocol work,
v18 has now a total of 11 meta-commands related to the extended query
protocol. These were all listed under the "General" section of the
--help=commands output and are specialized, bloating the output
generated.
All these meta-commands are moved into a new section called "Extended
Query Protocol", listed at the end of --help=commands.
This split has been suggested by Noah Misch.
Discussion: https://postgr.es/m/
20250415213450.1f.nmisch@google.com
Michael Paquier [Sat, 19 Apr 2025 23:16:57 +0000 (08:16 +0900)]
psql: Improve descriptions of \\flush[request] in --help
Noah has reported that the current wording was confusing compared to the
description of the underlying libpq routine. The new wording is from
me.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/
20250415213450.1f.nmisch@google.com
Michael Paquier [Sat, 19 Apr 2025 23:15:39 +0000 (08:15 +0900)]
psql: Fix incorrect status code returned by \getresults
When an invalid number of results is requested for \getresults, the
status code returned by exec_command_getresults() was PSQL_CMD_SKIP_LINE
and not PSQL_CMD_ERROR.
This led to incorrect behaviors, with ON_ERROR_STOP for example.
Reported-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/
20250415213450.1f.nmisch@google.com
Tom Lane [Sat, 19 Apr 2025 20:37:42 +0000 (16:37 -0400)]
Be more wary of corrupt data in pageinspect's heap_page_items().
The original intent in heap_page_items() was to return nulls, not
throw an error or crash, if an item was sufficiently corrupt that
we couldn't safely extract data from it. However, commit
d6061f83a
utterly missed that memo, and not only put in an un-length-checked
copy of the tuple's data section, but also managed to break the check
on sane nulls-bitmap length. Either mistake could possibly lead to
a SIGSEGV crash if the tuple is corrupt.
Bug: #18896
Reported-by: Dmitry Kovalenko <d.kovalenko@postgrespro.ru>
Author: Dmitry Kovalenko <d.kovalenko@postgrespro.ru>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/18896-
add267b8e06663e3@postgresql.org
Backpatch-through: 13
Michael Paquier [Sat, 19 Apr 2025 10:17:42 +0000 (19:17 +0900)]
Fix typos and grammar in the code
The large majority of these have been introduced by recent commits done
in the v18 development cycle.
Author: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/
9a7763ab-5252-429d-a943-
b28941e0e28b@gmail.com
Michael Paquier [Sat, 19 Apr 2025 09:53:35 +0000 (18:53 +0900)]
Rename injection points used in AIO tests
The format of the injection point names used by the AIO code does not
match the existing naming convention used everywhere else in the code,
so let's be consistent. These points are used in test_aio.
Reviewed-by: Hayato Kuroda <kuroda.hayato@fujitsu.com>
Discussion: https://postgr.es/m/Z_yTB80bdu1sYDqJ@paquier.xyz
Fujii Masao [Fri, 18 Apr 2025 09:35:40 +0000 (18:35 +0900)]
Make pg_upgrade log message with control file path translatable.
Commit
173c97812ff replaced the hardcoded "global/pg_control" in pg_upgrade
log message with a string literal concatenation of XLOG_CONTROL_FILE macro.
However, this change made the message untranslatable.
This commit fixes the issue by using %s with XLOG_CONTROL_FILE instead of
that literal concatenation, allowing the message to be translated properly.
It also wraps the file path in double quotes for consistency with similar
log messages.
Author: Kyotaro Horiguchi <horikyota.ntt@gmail.com>
Reviewed-by: Masao Fujii <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/
20250407.155546.
2129693791769531891.horikyota.ntt@gmail.com
Tatsuo Ishii [Fri, 18 Apr 2025 00:38:46 +0000 (09:38 +0900)]
Doc: fix missing comma at the end of a line.
Backpatch to 17, where the line was added.
Reported by Noboru Saito while he was working on translating the file
into Japanese.
Discussion: https://postgr.es/m/
20250417.203047.
1321297410457834775.ishii%40postgresql.org
Reported-by: Noboru Saito <noborusai@gmail.com>
Reviewed-by: Daniel Gustafs <daniel@yesql.se>
Backpatch-through: 17
David Rowley [Fri, 18 Apr 2025 00:15:08 +0000 (12:15 +1200)]
Fixup various older misuses of appendPQExpBuffer
Use appendPQExpBufferStr when there are no parameters and
appendPQExpBufferChar when the string length is 1.
Unlike
3fae25cbb, which fixed this issue for code that was new to v18,
this one fixes up instances which exist in the backbranches. We've
historically tried to maintain this standard and if we're going to
continue doing that, then we won't be doing that selectively based on
when the code was introduced. Now seems like a good time to flush out the
existing misuses. Waiting until v19 just prolongs their existence in
terms of released versions that the misuses exist in.
Author: David Rowley <drowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvoARMvPeXTTC0HnpARBHn-WgVstc8XFCyMGOzvgu_1HvQ@mail.gmail.com
David Rowley [Thu, 17 Apr 2025 21:04:28 +0000 (09:04 +1200)]
Make levels 1-based in pg_log_backend_memory_contexts()
Both pg_get_process_memory_contexts() and pg_backend_memory_contexts
have 1-based levels, whereas pg_log_backend_memory_contexts() was using
0-based levels. Align these.
This results in slightly saner behavior from MemoryContextStatsDetail()
in regards to the max_level. Previously it would stop at 1 level before
the maximum requested level rather than at that level.
Reported-by: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Author: Atsushi Torikoshi <torikoshia@oss.nttdata.com>
Author: David Rowley <drowleyml@gmail.com
Reviewed-by: Melih Mutlu <m.melihmutlu@gmail.com>
Reviewed-by: Rahila Syed <rahilasyed90@gmail.com>
Discussion: https://postgr.es/m/
395ea5d4fe190480efa95bf533485c70@oss.nttdata.com
Tom Lane [Thu, 17 Apr 2025 20:47:04 +0000 (16:47 -0400)]
Suppress "may be used uninitialized" warnings from older compilers.
The "children" list won't be used until "got_children" has been set
true, but older compilers don't get that; about half a dozen
buildfarm animals are warning about this. Issue added by
11ff192b5.
While here, improve slightly-shaky grammar in comment.
Discussion: https://postgr.es/m/
2057835.
1744833309@sss.pgh.pa.us
Tom Lane [Thu, 17 Apr 2025 20:33:21 +0000 (16:33 -0400)]
Portability fix: isdigit() must be passed an unsigned char.
Oversight in commit
40b9c2701, per buildfarm member mamba.
Tom Lane [Thu, 17 Apr 2025 16:56:40 +0000 (12:56 -0400)]
Cache typlens of a SQL function's input arguments.
This gets rid of repetitive get_typlen calls in postquel_sub_params,
which show up as costing a few percent of the runtime in simple test
cases (more with more parameters).
In combination with the preceding patches, this gets us most of the
way back down to the amount of per-call overhead that functions.c
had before commit
0dca5d68d. There are some more things that could
be done, but this seems like an okay place to stop for v18.
Tom Lane [Thu, 17 Apr 2025 16:56:31 +0000 (12:56 -0400)]
Make SQLFunctionCache long-lived again.
At this point, the only data structures we allocate directly in
fcontext are the SQLFunctionCache struct itself, the ParamListInfo
struct, and the execution_state array, all of which are small and
perfectly capable of being re-used across executions of the same
FmgrInfo. Hence, let's give them the same lifespan as the FmgrInfo.
This step gets rid of the separate SQLFunctionLink struct and makes
fn_extra point to SQLFunctionCache again. We also get rid of the
separate fcontext memory context and allocate these items directly
in fn_mcxt.
For notational simplicity, SQLFunctionCache still has an fcontext
field, but it's just a copy of fn_mcxt.
The motivation for this is to allow these structures to live as
long as the FmgrInfo and be re-used across calls, restoring the
original design without its propensity for memory leaks. This
gets rid of some per-call overhead that we added in
0dca5d68d.
We also make an effort to re-use the JunkFilter and result slot.
Those might need to change if the function definition changes,
so we compromise by rebuilding them if the cached plan changes.
This also moves the tuplestore into fn_mcxt so that it can be
re-used across calls, again undoing a change made in
0dca5d68d.
Tom Lane [Thu, 17 Apr 2025 16:56:21 +0000 (12:56 -0400)]
Split some storage out to separate subcontexts of fcontext.
Put the JunkFilter and its result slot (and thence also
some subsidiary data such as the result tupledesc) into a
separate subcontext "jfcontext". This doesn't accomplish
a lot at this point, because we make a new JunkFilter each
time through the SQL function. However, the plan is to make
the fcontext long-lived, and that raises the possibility
that we'll need a new JunkFilter because the plan for the
result-generating query changes. A separate context makes
it easy to free the obsoleted data when that happens.
Also, instead of always running the sub-executor in fcontext,
make a separate context for it if we're doing lazy eval of
a SRF, and otherwise just run it inside CurrentMemoryContext.
Tom Lane [Thu, 17 Apr 2025 16:56:08 +0000 (12:56 -0400)]
Make functions.c mostly run in a short-lived memory context.
Previously, much of this code ran with CurrentMemoryContext set
to be the function's fcontext, so that we tended to leak a lot of
stuff there. Commit
0dca5d68d dealt with that by releasing the
fcontext at the completion of each SQL function call, but we'd
like to go back to the previous approach of allowing the fcontext
to be query-lifespan. To control the leakage problem, rearrange
the code so that we mostly run in the memory context that fmgr_sql
is called in (which we expect to be short-lived). Notably, this
means that parsing/planning is all done in the short-lived context
and doesn't leak cruft into fcontext.
This patch also fixes the allocation of execution_state records
so that we don't leak them across executions. I set that up
with a re-usable array that contains at least as many
execution_state structs as we need for the current querytree.
The chain structure is still there, but it's not really doing
much for us, and maybe somebody will be motivated to get rid
of it. I'm not though.
This incidentally also moves the call of BlessTupleDesc to be
with the code that creates the JunkFilter. That doesn't make
much difference now, but a later patch will reduce the number
of times the JunkFilter gets made, and we needn't bless the
results any more often than that.
We still leak a fair amount in fcontext, particularly when
executing utility statements, but that's material for a
separate patch step; the point here is only to get rid of
unintentional allocations in fcontext.
Tom Lane [Thu, 17 Apr 2025 16:55:58 +0000 (12:55 -0400)]
Minor performance improvement for SQL-language functions.
Late in the development of commit
0dca5d68d, I added a step to copy
the result tlist we extract from the cached final query, because
I was afraid that that might not last as long as the JunkFilter that
we're passing it off to. However, that turns out to cost a noticeable
number of cycles, and it's really quite unnecessary because the
JunkFilter will not examine that tlist after it's been created.
(ExecFindJunkAttribute would use it, but we don't use that function
on this JunkFilter.) Hence, remove the copy step. For safety,
reset the might-become-dangling jf_targetList pointer to NIL.
In passing, remove DR_sqlfunction.cxt, which we don't use anymore;
it's confusing because it's not entirely clear which context it
ought to point at.
Noah Misch [Thu, 17 Apr 2025 12:00:30 +0000 (05:00 -0700)]
Assert lack of hazardous buffer locks before possible catalog read.
Commit
0bada39c83a150079567a6e97b1a25a198f30ea3 fixed a bug of this kind,
which existed in all branches for six days before detection. While the
probability of reaching the trouble was low, the disruption was extreme. No
new backends could start, and service restoration needed an immediate
shutdown. Hence, add this to catch the next bug like it.
The new check in RelationIdGetRelation() suffices to make autovacuum detect
the bug in commit
243e9b40f1b2dd09d6e5bf91ebf6e822a2cd3704 that led to commit
0bada39. This also checks in a number of similar places. It replaces each
Assert(IsTransactionState()) that pertained to a conditional catalog read.
No back-patch for now, but a back-patch of commit
243e9b4 should back-patch
this, too. A back-patch could omit the src/test/regress changes, since back
branches won't gain new index columns.
Reported-by: Alexander Lakhin <exclusion@gmail.com>
Discussion: https://postgr.es/m/
20250410191830.0e.nmisch@google.com
Discussion: https://postgr.es/m/
10ec0bc3-5933-1189-6bb8-
5dec4114558e@gmail.com
Daniel Gustafsson [Thu, 17 Apr 2025 10:58:00 +0000 (12:58 +0200)]
pg_dump: Set private_date pointer to NULL in callback
The end callback for ZStandard compression frees the private_data
but didn't set the pointer to NULL after freeing. This is not a
bug as the code is right now, since nothing is dereferencing the
pointer upon returning from the callback but it is good practice
to do.
Author: Alexander Kuznetsov <kuznetsovam@altlinux.org>
Reviewed-by: Daniel Gustafsson <daniel@yesql.se>
Reviewed-by: Ashutosh Bapat <ashutosh.bapat.oss@gmail.com>
Reviewed-by: Michael Paquier <michael@paquier.xyz>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Discussion: https://postgr.es/m/
efaee52b-9550-44ca-8633-
ea86076b3283@altlinux.org
Fujii Masao [Thu, 17 Apr 2025 00:52:47 +0000 (09:52 +0900)]
pg_dump: Fix incorrect archive format shown in error message.
In pg_dump and pg_restore, _allocAH() calls _discoverArchiveFormat() to
determine the archive format when the input format is unknown one.
If the input or discovered format is unrecognized, it reports an error
including the archive format number.
If discovered format is unrecognized, its number should be shown in
the error message. But previously the error message mistakenly showed
the originally requested format number (i.e., unknown one) instead of
the discovered one, due to referencing the wrong variable in the error
message.
This commit corrects the issue by using the appropriate variable in
the error message.
This fix has no practical impact since _discoverArchiveFormat() never
returns an unrecognized format and that error mesasge is actually
never output. Therefore, while the issue exists in back branches,
it's not worth the trouble and buildfarm cycles to back-patch.
So this fix is applied only to the master branch.
Author: Mahendra Singh Thalor <mahi6run@gmail.com>
Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us>
Reviewed-by: Fujii Masao <masao.fujii@gmail.com>
Discussion: https://postgr.es/m/CAKYtNAqu+N-Ab2Fq6wzNSOm_-0N-BMneanYNV1+6kFDXjva1Eg@mail.gmail.com
Jeff Davis [Wed, 16 Apr 2025 23:46:22 +0000 (16:46 -0700)]
Another unintentional behavior change in commit
e9931bfb75.
Reported-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/
20250412123430.8c.nmisch@google.com
Jeff Davis [Wed, 16 Apr 2025 23:46:16 +0000 (16:46 -0700)]
Improve comment in regc_pg_locale.c.
Reported-by: Noah Misch <noah@leadboat.com>
Reviewed-by: Noah Misch <noah@leadboat.com>
Discussion: https://postgr.es/m/
20250412123430.8c.nmisch@google.com
David Rowley [Wed, 16 Apr 2025 23:37:55 +0000 (11:37 +1200)]
Fixup various new-to-v18 usages of appendPQExpBuffer
Use appendPQExpBufferStr when there are no parameters and
appendPQExpBufferChar when the string length is 1.
Author: David Rowley <drowleyml@gmail.com>
Discussion: https://postgr.es/m/CAApHDvoARMvPeXTTC0HnpARBHn-WgVstc8XFCyMGOzvgu_1HvQ@mail.gmail.com