-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
Search before asking
- I searched in the issues and found nothing similar.
Paimon version
master
Compute Engine
all
Minimal reproduce step
Summary
When a branch is created based on a tag, deleting that tag can cause the branch's base snapshot to be expired during snapshot expiration operations, making the branch unqueryable.
Currently, Paimon has a protection mechanism that prevents snapshots referenced by tags from being expired (implemented in ExpireSnapshotsImpl.java). However, there's no corresponding protection for branches that reference tags.
The issue occurs in the following scenario:
- Create a tag pointing to a specific snapshot
- Create a branch based on this tag
- Delete the tag
- Run snapshot expiration
- The base snapshot of the branch gets expired because it's no longer protected by the tag
- The branch becomes unqueryable
What doesn't meet your expectations?
base snapshot not be deleted
Anything else?
Proposed Solutions
Solution 1: Tag Deletion Protection (Preferred)
Add a protection mechanism when deleting tags to prevent deletion if any branches reference the tag.
Implementation approach:
- Modify
TagManager.deleteTag()method - Before deleting a tag, check if any branches reference this tag
- If branches exist, either:
- Reject the deletion with an error message
- Require a force flag to proceed with deletion
Advantages:
- Prevents the problem at the source
- Clear error message to users
- Maintains data integrity
Solution 2: Branch-aware Snapshot Expiration
Modify the snapshot expiration logic to consider branches when determining which snapshots to protect.
Implementation approach:
- Modify
ExpireSnapshotsImpl.expireUntil()method - Add branch manager to collect all branch base snapshots
- Include branch-referenced snapshots in the skipping set
Advantages:
- More comprehensive protection
- Handles edge cases where branches might reference snapshots directly
Are you willing to submit a PR?
- I'm willing to submit a PR!