Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add new function ptrack_get_change_file_stat(start_lsn pg_lsn)
  • Loading branch information
ololobus committed May 12, 2021
commit 3026be92c398eeeb7bc8edf65ca0deef25c82c17
8 changes: 7 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,9 @@ To disable `ptrack` and clean up all remaining service files set `ptrack.map_siz

* ptrack_version() — returns ptrack version string.
* ptrack_init_lsn() — returns LSN of the last ptrack map initialization.
* ptrack_get_pagemapset('LSN') — returns a set of changed data files with bitmaps of changed blocks since specified LSN.
* ptrack_get_pagemapset(start_lsn pg_lsn) — returns a set of changed data files with bitmaps of changed blocks since specified `start_lsn`.
* ptrack_get_change_stat(start_lsn pg_lsn) — returns statistic of changes (number of files, pages and size in MB) since specified `start_lsn`.
* ptrack_get_change_file_stat(start_lsn pg_lsn) — returns per file statistic of changes (number of pages and size in MB) since specified `start_lsn`.

Usage example:

Expand Down Expand Up @@ -102,6 +104,10 @@ Usually, you have to only install new version of `ptrack` and do `ALTER EXTENSIO
* Do `ALTER EXTENSION 'ptrack' UPDATE;`.
* Restart your server.

#### Upgrading from 2.1.* to 2.2.*:

Since version 2.2 we use a different algorithm for tracking changed pages. Thus, data recorded in the `ptrack.map` using pre 2.2 versions of `ptrack` is incompatible with newer versions. After extension upgrade and server restart old `ptrack.map` will be discarded with `WARNING` and initialized from the scratch.

## Limitations

1. You can only use `ptrack` safely with `wal_level >= 'replica'`. Otherwise, you can lose tracking of some changes if crash-recovery occurs, since [certain commands are designed not to write WAL at all if wal_level is minimal](https://www.postgresql.org/docs/12/populate.html#POPULATE-PITR), but we only durably flush `ptrack` map at checkpoint time.
Expand Down
25 changes: 25 additions & 0 deletions ptrack--2.1--2.2.sql
Original file line number Diff line number Diff line change
Expand Up @@ -27,3 +27,28 @@ BEGIN
FROM ptrack_get_pagemapset(start_lsn)) s;
END
$func$ LANGUAGE plpgsql;

CREATE FUNCTION ptrack_get_change_file_stat(start_lsn pg_lsn)
RETURNS TABLE (
file_path text,
pages int,
"size, MB" numeric
) AS
$func$
DECLARE
block_size bigint;
BEGIN
block_size := (SELECT setting FROM pg_settings WHERE name = 'block_size');

RETURN QUERY
SELECT s.path,
changed_pages,
block_size*changed_pages/(1024.0*1024)
FROM
(SELECT path,
length(replace(right((pagemap)::text, -1)::varbit::text, '0', ''))
AS changed_pages
FROM ptrack_get_pagemapset(start_lsn)) s
ORDER BY (changed_pages, s.path) DESC;
END
$func$ LANGUAGE plpgsql;
5 changes: 4 additions & 1 deletion t/001_basic.pl
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
use TestLib;
use Test::More;

plan tests => 24;
plan tests => 25;

my $node;
my $res;
Expand Down Expand Up @@ -119,6 +119,9 @@
$res_stdout = $node->safe_psql("postgres", "SELECT pages FROM ptrack_get_change_stat('$flush_lsn')");
is($res_stdout > 0, 1, 'should be able to get aggregated stats of changes');

$res_stdout = $node->safe_psql("postgres", "SELECT count(*) FROM ptrack_get_change_file_stat('$flush_lsn')");
is($res_stdout > 0, 1, 'should be able to get per file stats of changes');

# We should be able to change ptrack map size (but loose all changes)
$node->append_conf(
'postgresql.conf', q{
Expand Down