Skip to content

chore: enqueue pending_prune_days on inserts#1117

Merged
MicBun merged 3 commits intomainfrom
chore/insert-pending
Aug 20, 2025
Merged

chore: enqueue pending_prune_days on inserts#1117
MicBun merged 3 commits intomainfrom
chore/insert-pending

Conversation

@outerlook
Copy link
Contributor

Description

  • Add idempotent enqueue of (stream_ref, day_index) into pending_prune_days on primitive and truflation batch inserts
  • Ignore zero-value writes when enqueuing (keeps queue focused on effective changes)
  • Consolidate to a single private helper helper_enqueue_prune_days(stream_refs INT[], event_times INT8[], values NUMERIC[])
  • Make insert_record a thin wrapper over insert_records to unify behavior
  • Keep schema constraints enforced: (stream_ref, day_index) PK; day_index >= 0

Related Problem

How Has This Been Tested?

  • Added focused tests verifying presence and idempotency of queued days for both paths:
    • tests/streams/pending_prune_days_test.go
    • Covers primitive single/batch and truflation_insert_records
  • Tests pass locally

Reviewer Pointers (files of interest)

  • internal/migrations/901-utilities.sql: new helper_enqueue_prune_days (filtered) and cleanup
  • internal/migrations/003-primitive-insertion.sql: enqueue on primitive insert_records; insert_record wraps
  • internal/migrations/009-truflation-query.sql: enqueue on truflation batch path
  • internal/migrations/normalization_migration/02_insert_actions.sql: enqueue for migration-compatible path
  • tests/streams/pending_prune_days_test.go: minimal high-ROI coverage

- Changed the visibility of the `auto_digest` action from private to public in the `020-digest-actions.sql` migration script, allowing it to be called directly from the CLI for verification purposes.
- Refactored the `insert_record` and `insert_records` actions to utilize a new `insert_records` function for batch processing.
- Introduced `helper_enqueue_prune_days` to enqueue days for pruning, ensuring idempotency and filtering out zero values.
- Added tests to verify the correct population of `pending_prune_days` during record insertion, maintaining data integrity across multiple inserts.
- Replaced the usage of `get_stream_ids` with `$stream_refs` in the `insert_records` action to ensure correct stream reference handling during pruning.
- Added a condition to the `helper_enqueue_prune_days` function to filter out negative event times, enhancing data integrity during pruning operations.
@outerlook outerlook self-assigned this Aug 20, 2025
@coderabbitai
Copy link

coderabbitai bot commented Aug 20, 2025

Warning

Rate limit exceeded

@outerlook has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 28 minutes and 9 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between 1c67faa and 573ba0f.

📒 Files selected for processing (6)
  • internal/migrations/003-primitive-insertion.sql (2 hunks)
  • internal/migrations/009-truflation-query.sql (1 hunks)
  • internal/migrations/020-digest-actions.sql (1 hunks)
  • internal/migrations/901-utilities.sql (1 hunks)
  • internal/migrations/normalization_migration/02_insert_actions.sql (2 hunks)
  • tests/streams/pending_prune_days_test.go (1 hunks)
✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch chore/insert-pending

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@MicBun MicBun mentioned this pull request Aug 20, 2025
@MicBun MicBun merged commit 14329e2 into main Aug 20, 2025
9 of 13 checks passed
@MicBun MicBun deleted the chore/insert-pending branch August 20, 2025 16:31
@holdex
Copy link

holdex bot commented Aug 20, 2025

Time Submission Status

Member Status Time Action Last Update
@outerlook ❌ Missing - ⚠️ Submit time -
MicBun ✅ Submitted 20min Update time Aug 20, 2025, 4:35 PM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Problem: Record insertion missing digest queue integration

2 participants