A little tool i vibecoded for analyzing the data I push to splunk to save money on licensing.

Idea is: ncdu for JSON. This tool lets you explore your json structure and it's size in a nice TUI. The tool tries to aggregate events when you pass a [{object},{object}] style json, it will aggregate the objects and measure the size of individual fields, so this tool may not be the best for you if your events highly differ in field naming. If you push lots of events with consistent field naming, then try it out!

Depending on the tool you plan to ingest your data into, you may want to use the --include-keys flag depending on if the tool stores the key names multiple times, if not then keep it off.

JSON/JSONL exploration tool for Splunk data storage analysis

Usage: jdu [OPTIONS] <PATH>

Arguments:
  <PATH>  Path to the JSON/JSONL file

Options:
  -f, --file-type <FILE_TYPE>  File type (json or jsonl) [default: auto]
  -k, --include-keys           Include key names in size calculation (default: false)
  -s, --strings-only           Treat all values as strings for size calculation (default: false)
  -h, --help                   Print help
  -V, --version                Print version