docs: add per-column Parquet encoding and compression documentation#384
docs: add per-column Parquet encoding and compression documentation#384
Conversation
RaphDal
added
documentation
|
🚀 Build success! Latest successful preview: https://preview-384--questdb-documentation.netlify.app/docs/ Commit SHA: 6c99710
|
| the server-wide compression codec | ||
| (`cairo.partition.encoder.parquet.compression.codec`). | ||
|
|
||
| #### Supported encodings |
There was a problem hiding this comment.
Should we explain what each encoding is good for? Even if just a link to an authoritative 3rd party docs. Without that, I have no idea why I would choose delta_binary_packed, as it is not used by default for numbers, so no idea when it can be convenient
There was a problem hiding this comment.
Yes, definitely!
942dfc3 adds a reference to the official link and a small summary per encoding.
| `rle_dictionary` for SYMBOL and VARCHAR, `delta_length_byte_array` for STRING | ||
| and BINARY, and `plain` for everything else. | ||
|
|
||
| #### Supported compression codecs |
There was a problem hiding this comment.
same thing as above. We should probably tell the good/bad for each method, or link to somewhere where this is explained
2 participants
Tandem PR for questdb/questdb#6843.
This pull request introduces comprehensive documentation and configuration support for per-column Parquet encoding and compression in QuestDB, along with a new server property to control Parquet page compression efficiency. The changes add detailed SQL syntax, configuration options, and usage examples for both table creation and schema alteration, and update relevant documentation and diagrams to reflect these enhancements.
Per-column Parquet encoding and compression support:
ENCODINGandCOMPRESSIONinCREATE TABLEstatements, including supported encodings/codecs, syntax diagrams, and usage examples. [1] [2]ALTER TABLE ALTER COLUMN SET/DROP PARQUET ENCODING/COMPRESSION, allowing users to modify or reset per-column Parquet settings on existing tables. [1] [2]SHOW CREATE TABLEdocumentation to display per-column Parquet overrides in table definitions.Parquet compression configuration:
cairo.partition.encoder.parquet.min.compression.ratioproperty, which determines whether a compressed Parquet page is stored compressed or uncompressed based on its compression ratio. Provided usage guidance and default values. [1] [2]These updates make it easier for users to fine-tune Parquet export behavior and understand the available configuration options.