Skip to content

Implement comprehensive profile size limits across ingestion endpoints#4734

Merged
simonswine merged 13 commits intomainfrom
20260108_limit-gzip-compression-gain
Jan 9, 2026
Merged

Implement comprehensive profile size limits across ingestion endpoints#4734
simonswine merged 13 commits intomainfrom
20260108_limit-gzip-compression-gain

Conversation

@simonswine
Copy link
Contributor

@simonswine simonswine commented Jan 8, 2026

Summary

This PR implements comprehensive size limit enforcement across all profile ingestion endpoints to protect against resource exhaustion from excessively large profiles and "gzip bomb" payloads.

Key Changes

  • Decompression limits: Added RawFromBytesWithLimit() to enforce decompressed profile size limits, preventing small compressed payloads from expanding to huge sizes
  • Push endpoint: Enforce per-profile and total batched request size limits using MaxProfileSizeBytes and IngestionBodyLimitBytes
  • OTLP endpoint: Enforce decompressed body size limits for OTLP ingestion with proper HTTP 413 responses
  • AdHoc profiler: Validate profile sizes on upload with human-readable error messages
  • Symbolizer: Added MaxSymbolSizeBytes configuration (default: 512MB) to limit symbol file sizes from debuginfod
  • Metrics: Track discarded profiles/bytes per tenant when limits are exceeded using validation.DiscardedBytes and validation.DiscardedProfiles

Testing

Added comprehensive test coverage for all endpoints including size limit enforcement scenarios.

Configuration

New flag: --validation.symbolizer.max-symbol-size-bytes (default: 512MB)

All limits are backwards compatible and only enforce when configured > 0.

@simonswine simonswine changed the title 20260108 limit gzip compression gain Implement comprehensive profile size limits across ingestion endpoints Jan 8, 2026
@simonswine simonswine marked this pull request as ready for review January 8, 2026 10:25
Copy link
Contributor

@aleks-p aleks-p left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for adding tests!

@simonswine simonswine merged commit 4523fd1 into main Jan 9, 2026
20 checks passed
@simonswine simonswine deleted the 20260108_limit-gzip-compression-gain branch January 9, 2026 08:43
simonswine added a commit that referenced this pull request Jan 9, 2026
simonswine added a commit that referenced this pull request Jan 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants