Skip to content

Commit ec9549d

Browse files
committed
Add role_id, database_id, leader_pid and is_regular_backend columns
Also add a GUC to handle all "sampling dimensions"
1 parent f96fb9e commit ec9549d

File tree

10 files changed

+677
-258
lines changed

10 files changed

+677
-258
lines changed

Makefile

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ MODULE_big = pg_wait_sampling
44
OBJS = pg_wait_sampling.o collector.o
55

66
EXTENSION = pg_wait_sampling
7-
DATA = pg_wait_sampling--1.1.sql pg_wait_sampling--1.0--1.1.sql
7+
DATA = pg_wait_sampling--1.1.sql pg_wait_sampling--1.0--1.1.sql pg_wait_sampling--1.1--1.2.sql
88

99
REGRESS = load queries
1010

README.md

Lines changed: 58 additions & 33 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,9 @@ When `pg_wait_sampling` is enabled, it collects two kinds of statistics.
3030
recent samples depending on history size (configurable). Assuming there is
3131
a client who periodically read this history and dump it somewhere, user
3232
can have continuous history.
33-
* Waits profile. It's implemented as in-memory hash table where count
34-
of samples are accumulated per each process and each wait event
35-
(and each query with `pg_stat_statements`). This hash
33+
* Waits profile. It's implemented as in-memory hash table where samples
34+
are accumulated and can be grouped by process, wait event, query, user and/or
35+
database (and then joined by queryid with `pg_stat_statements`). This hash
3636
table can be reset by user request. Assuming there is a client who
3737
periodically dumps profile and resets it, user can have statistics of
3838
intensivity of wait events among time.
@@ -98,53 +98,67 @@ Usage
9898
`pg_wait_sampling` interacts with user by set of views and functions.
9999

100100
`pg_wait_sampling_current` view – information about current wait events for
101-
all processed including background workers.
102-
103-
| Column name | Column type | Description |
104-
| ----------- | ----------- | ----------------------- |
105-
| pid | int4 | Id of process |
106-
| event_type | text | Name of wait event type |
107-
| event | text | Name of wait event |
108-
| queryid | int8 | Id of query |
101+
all processes including background workers.
102+
103+
| Column name | Column type | Description |
104+
| ------------------- | ----------- | --------------------------- |
105+
| pid | int4 | Id of process |
106+
| event_type | text | Name of wait event type |
107+
| event | text | Name of wait event |
108+
| queryid | int8 | Id of query |
109+
| role_id | int4 | Id of role |
110+
| database_id | int4 | Id of database |
111+
| leader_pid | int4 | Id of parallel query leader |
112+
| is_regular_backend | bool | Is backend or worker |
109113

110114
`pg_wait_sampling_get_current(pid int4)` returns the same table for single given
111115
process.
112116

113117
`pg_wait_sampling_history` view – history of wait events obtained by sampling into
114118
in-memory ring buffer.
115119

116-
| Column name | Column type | Description |
117-
| ----------- | ----------- | ----------------------- |
118-
| pid | int4 | Id of process |
119-
| ts | timestamptz | Sample timestamp |
120-
| event_type | text | Name of wait event type |
121-
| event | text | Name of wait event |
122-
| queryid | int8 | Id of query |
120+
| Column name | Column type | Description |
121+
| ------------------- | ----------- | --------------------------- |
122+
| pid | int4 | Id of process |
123+
| event_type | text | Name of wait event type |
124+
| event | text | Name of wait event |
125+
| queryid | int8 | Id of query |
126+
| role_id | int4 | Id of role |
127+
| database_id | int4 | Id of database |
128+
| leader_pid | int4 | Id of parallel query leader |
129+
| is_regular_backend | bool | Is backend or worker |
130+
| ts | timestamptz | Sample timestamp |
123131

124132
`pg_wait_sampling_profile` view – profile of wait events obtained by sampling into
125133
in-memory hash table.
126134

127-
| Column name | Column type | Description |
128-
| ----------- | ----------- | ----------------------- |
129-
| pid | int4 | Id of process |
130-
| event_type | text | Name of wait event type |
131-
| event | text | Name of wait event |
132-
| queryid | int8 | Id of query |
133-
| count | text | Count of samples |
135+
| Column name | Column type | Description |
136+
| ------------------- | ----------- | --------------------------- |
137+
| pid | int4 | Id of process |
138+
| event_type | text | Name of wait event type |
139+
| event | text | Name of wait event |
140+
| queryid | int8 | Id of query |
141+
| role_id | int4 | Id of role |
142+
| database_id | int4 | Id of database |
143+
| leader_pid | int4 | Id of parallel query leader |
144+
| is_regular_backend | bool | Is backend or worker |
145+
| count | text | Count of samples |
134146

135147
`pg_wait_sampling_reset_profile()` function resets the profile.
136148

137149
The work of wait event statistics collector worker is controlled by following
138150
GUCs.
139151

140-
| Parameter name | Data type | Description | Default value |
141-
|----------------------------------| --------- |---------------------------------------------|--------------:|
142-
| pg_wait_sampling.history_size | int4 | Size of history in-memory ring buffer | 5000 |
143-
| pg_wait_sampling.history_period | int4 | Period for history sampling in milliseconds | 10 |
144-
| pg_wait_sampling.profile_period | int4 | Period for profile sampling in milliseconds | 10 |
145-
| pg_wait_sampling.profile_pid | bool | Whether profile should be per pid | true |
146-
| pg_wait_sampling.profile_queries | enum | Whether profile should be per query | top |
147-
| pg_wait_sampling.sample_cpu | bool | Whether on CPU backends should be sampled | true |
152+
| Parameter name | Data type | Description | Default value |
153+
|-------------------------------------| --------- |---------------------------------------------|-----------------------|
154+
| pg_wait_sampling.history_size | int4 | Size of history in-memory ring buffer | 5000 |
155+
| pg_wait_sampling.history_period | int4 | Period for history sampling in milliseconds | 10 |
156+
| pg_wait_sampling.profile_period | int4 | Period for profile sampling in milliseconds | 10 |
157+
| pg_wait_sampling.profile_pid | bool | Whether profile should be per pid | true |
158+
| pg_wait_sampling.profile_queries | enum | Whether profile should be per query | top |
159+
| pg_wait_sampling.sample_cpu | bool | Whether on CPU backends should be sampled | true |
160+
| pg_wait_sampling.history_dimensions | text | Columns that are sampled for history | 'pid, event, queryid' |
161+
| pg_wait_sampling.profile_dimensions | text | Columns that are sampled for profile | 'pid, event, queryid' |
148162

149163
If `pg_wait_sampling.profile_pid` is set to false, sampling profile wouldn't be
150164
collected in per-process manner. In this case the value of pid could would
@@ -158,10 +172,21 @@ If `pg_wait_sampling.sample_cpu` is set to true then processes that are not
158172
waiting on anything are also sampled. The wait event columns for such processes
159173
will be NULL.
160174

175+
`pg_wait_sampling.history_dimenstions` and `pg_wait_sampling.profile_dimensions`
176+
determine what columns will be sampled in `history/profile` views.
177+
Allowed values are `all`, `pid`, `event`, `query_id`, `role_id`,
178+
`database_id`, `leader_pid` and any combination of column names.
179+
`event` turns on and off both event and event_type columns.
180+
`all` cannot be used together with any other values and must be used alone.
181+
161182
Values of these GUC variables can be changed only in config file or with ALTER SYSTEM.
162183
Then you need to reload server's configuration (such as with pg_reload_conf function)
163184
for changes to take effect.
164185

186+
> [!WARNING]
187+
> Using `pg_reload_conf` will reset `pg_wait_sampling_history` and
188+
> `pg_wait_sampling_profile` views if new dimensions differ from old ones.
189+
165190
See
166191
[PostgreSQL documentation](http://www.postgresql.org/docs/devel/static/monitoring-stats.html#WAIT-EVENT-TABLE)
167192
for list of possible wait events.

collector.c

Lines changed: 88 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,9 @@
3232

3333
static volatile sig_atomic_t shutdown_requested = false;
3434

35+
int saved_profile_dimensions;
36+
int saved_history_dimensions;
37+
3538
static void handle_sigterm(SIGNAL_ARGS);
3639

3740
/*
@@ -61,7 +64,8 @@ pgws_register_wait_collector(void)
6164
static void
6265
alloc_history(History *observations, int count)
6366
{
64-
observations->items = (HistoryItem *) palloc0(sizeof(HistoryItem) * count);
67+
saved_history_dimensions = pgws_history_dimensions;
68+
observations->samples = (Sample *) palloc0(sizeof(Sample) * count);
6569
observations->index = 0;
6670
observations->count = count;
6771
observations->wraparound = false;
@@ -73,13 +77,13 @@ alloc_history(History *observations, int count)
7377
static void
7478
realloc_history(History *observations, int count)
7579
{
76-
HistoryItem *newitems;
80+
Sample *newitems;
7781
int copyCount,
7882
i,
7983
j;
8084

8185
/* Allocate new array for history */
82-
newitems = (HistoryItem *) palloc0(sizeof(HistoryItem) * count);
86+
newitems = (Sample *) palloc0(sizeof(Sample) * count);
8387

8488
/* Copy entries from old array to the new */
8589
if (observations->wraparound)
@@ -98,14 +102,14 @@ realloc_history(History *observations, int count)
98102
{
99103
if (j >= observations->count)
100104
j = 0;
101-
memcpy(&newitems[i], &observations->items[j], sizeof(HistoryItem));
105+
memcpy(&newitems[i], &observations->samples[j], sizeof(Sample));
102106
i++;
103107
j++;
104108
}
105109

106110
/* Switch to new history array */
107-
pfree(observations->items);
108-
observations->items = newitems;
111+
pfree(observations->samples);
112+
observations->samples = newitems;
109113
observations->index = copyCount;
110114
observations->count = count;
111115
observations->wraparound = false;
@@ -125,22 +129,55 @@ handle_sigterm(SIGNAL_ARGS)
125129
/*
126130
* Get next item of history with rotation.
127131
*/
128-
static HistoryItem *
132+
static Sample *
129133
get_next_observation(History *observations)
130134
{
131-
HistoryItem *result;
135+
Sample *result;
132136

133137
/* Check for wraparound */
134138
if (observations->index >= observations->count)
135139
{
136140
observations->index = 0;
137141
observations->wraparound = true;
138142
}
139-
result = &observations->items[observations->index];
143+
result = &observations->samples[observations->index];
140144
observations->index++;
141145
return result;
142146
}
143147

148+
void
149+
fill_sample(Sample *sample, PGPROC *proc, int pid, uint32 wait_event_info,
150+
uint64 queryId, int dimensions_mask)
151+
{
152+
Oid role_id = proc->roleId;
153+
Oid database_id = proc->databaseId;
154+
PGPROC *lockGroupLeader = proc->lockGroupLeader;
155+
#if PG_VERSION_NUM >= 180000
156+
bool is_regular_backend = proc->isRegularBackend;
157+
#else
158+
bool is_regular_backend = !proc->isBackgroundWorker;
159+
#endif
160+
161+
if (dimensions_mask & PGWS_DIMENSIONS_PID)
162+
sample->pid = pid;
163+
if (dimensions_mask & PGWS_DIMENSIONS_WAIT_EVENT)
164+
sample->wait_event_info = wait_event_info;
165+
if (pgws_profileQueries || (dimensions_mask & PGWD_DIMENSIONS_QUERY_ID))
166+
sample->queryId = queryId;
167+
/* Copy everything else we need from PGPROC */
168+
if (dimensions_mask & PGWS_DIMENSIONS_ROLE_ID)
169+
sample->role_id = role_id;
170+
if (dimensions_mask & PGWS_DIMENSIONS_DB_ID)
171+
sample->database_id = database_id;
172+
if (dimensions_mask & PGWS_DIMENSIONS_PARALLEL_LEADER_PID)
173+
sample->parallel_leader_pid = (lockGroupLeader &&
174+
lockGroupLeader->pid != pid ?
175+
lockGroupLeader->pid :
176+
0);
177+
if (dimensions_mask & PGWS_DIMENSIONS_IS_REGULAR_BE)
178+
sample->is_regular_backend = is_regular_backend;
179+
}
180+
144181
/*
145182
* Read current waits from backends and write them to history array
146183
* and/or profile hash.
@@ -162,37 +199,38 @@ probe_waits(History *observations, HTAB *profile_hash,
162199
LWLockAcquire(ProcArrayLock, LW_SHARED);
163200
for (i = 0; i < ProcGlobal->allProcCount; i++)
164201
{
165-
HistoryItem item,
166-
*observation;
167-
PGPROC *proc = &ProcGlobal->allProcs[i];
202+
/* We do not copy PGPROC since it is very big */
203+
PGPROC *proc = &ProcGlobal->allProcs[i];
204+
int pid;
205+
uint32 wait_event_info;
168206

169-
if (!pgws_should_sample_proc(proc, &item.pid, &item.wait_event_info))
207+
if (!pgws_should_sample_proc(proc, &pid, &wait_event_info))
170208
continue;
171209

172-
if (pgws_profileQueries)
173-
item.queryId = pgws_proc_queryids[i];
174-
else
175-
item.queryId = 0;
176-
177-
item.ts = ts;
178-
179210
/* Write to the history if needed */
180211
if (write_history)
181212
{
182-
observation = get_next_observation(observations);
183-
*observation = item;
213+
Sample *observation = get_next_observation(observations);
214+
memset(observation, 0, sizeof(Sample));
215+
fill_sample(observation, proc, pid, wait_event_info,
216+
pgws_proc_queryids[i], saved_history_dimensions);
217+
observation->ts = ts;
184218
}
185219

186220
/* Write to the profile if needed */
187221
if (write_profile)
188222
{
189-
ProfileItem *profileItem;
190-
bool found;
223+
Sample *profileItem;
224+
Sample key;
225+
bool found;
191226

227+
memset(&key, 0, sizeof(Sample));
228+
fill_sample(&key, proc, pid, wait_event_info,
229+
pgws_proc_queryids[i], saved_profile_dimensions);
192230
if (!profile_pid)
193-
item.pid = 0;
231+
key.pid = 0;
194232

195-
profileItem = (ProfileItem *) hash_search(profile_hash, &item, HASH_ENTER, &found);
233+
profileItem = (Sample *) hash_search(profile_hash, &key, HASH_ENTER, &found);
196234
if (found)
197235
profileItem->count++;
198236
else
@@ -229,8 +267,8 @@ send_history(History *observations, shm_mq_handle *mqh)
229267
for (i = 0; i < count; i++)
230268
{
231269
mq_result = shm_mq_send_compat(mqh,
232-
sizeof(HistoryItem),
233-
&observations->items[i],
270+
sizeof(Sample),
271+
&observations->samples[i],
234272
false,
235273
true);
236274
if (mq_result == SHM_MQ_DETACHED)
@@ -249,10 +287,10 @@ send_history(History *observations, shm_mq_handle *mqh)
249287
static void
250288
send_profile(HTAB *profile_hash, shm_mq_handle *mqh)
251289
{
252-
HASH_SEQ_STATUS scan_status;
253-
ProfileItem *item;
254-
Size count = hash_get_num_entries(profile_hash);
255-
shm_mq_result mq_result;
290+
HASH_SEQ_STATUS scan_status;
291+
Sample *sample;
292+
Size count = hash_get_num_entries(profile_hash);
293+
shm_mq_result mq_result;
256294

257295
/* Send array size first since receive_array expects this */
258296
mq_result = shm_mq_send_compat(mqh, sizeof(count), &count, false, true);
@@ -264,9 +302,9 @@ send_profile(HTAB *profile_hash, shm_mq_handle *mqh)
264302
return;
265303
}
266304
hash_seq_init(&scan_status, profile_hash);
267-
while ((item = (ProfileItem *) hash_seq_search(&scan_status)) != NULL)
305+
while ((sample = (Sample *) hash_seq_search(&scan_status)) != NULL)
268306
{
269-
mq_result = shm_mq_send_compat(mqh, sizeof(ProfileItem), item, false,
307+
mq_result = shm_mq_send_compat(mqh, sizeof(Sample), sample, false,
270308
true);
271309
if (mq_result == SHM_MQ_DETACHED)
272310
{
@@ -287,12 +325,10 @@ make_profile_hash()
287325
{
288326
HASHCTL hash_ctl;
289327

290-
if (pgws_profileQueries)
291-
hash_ctl.keysize = offsetof(ProfileItem, count);
292-
else
293-
hash_ctl.keysize = offsetof(ProfileItem, queryId);
294-
295-
hash_ctl.entrysize = sizeof(ProfileItem);
328+
saved_profile_dimensions = pgws_profile_dimensions;
329+
/* Fields that are not in dimensions mask are zero and are included in key */
330+
hash_ctl.keysize = offsetof(Sample, count);
331+
hash_ctl.entrysize = sizeof(Sample);
296332
return hash_create("Waits profile hash", 1024, &hash_ctl,
297333
HASH_ELEM | HASH_BLOBS);
298334
}
@@ -377,6 +413,18 @@ pgws_collector_main(Datum main_arg)
377413
{
378414
ConfigReloadPending = false;
379415
ProcessConfigFile(PGC_SIGHUP);
416+
417+
/* Reset profile and history if needed */
418+
if (pgws_history_dimensions != saved_history_dimensions)
419+
{
420+
pfree(observations.samples);
421+
alloc_history(&observations, pgws_historySize);
422+
}
423+
if (pgws_profile_dimensions != saved_profile_dimensions)
424+
{
425+
hash_destroy(profile_hash);
426+
profile_hash = make_profile_hash();
427+
}
380428
}
381429

382430
/* Calculate time to next sample for history or profile */

0 commit comments

Comments
 (0)