-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[fix][broker]Fixed an issue where sending chunk messages would be lost when no consumers were online #25077
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
[fix][broker]Fixed an issue where sending chunk messages would be lost when no consumers were online #25077
Conversation
…t when no consumers were online
|
@zjxxzjwang Please add the following content to your PR description and select a checkbox: |
lhotari
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for the PR @zjxxzjwang . Please add a test that demonstrates the broken behavior and shows that the changes in this PR fix the problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR fixes an issue where chunked messages would be lost when no consumers are online for a subscription. The fix ensures that when no consumer is available to receive messages, the messages are sent to the redelivery queue instead of being released, allowing them to be consumed when consumers come back online.
Key Changes
- Modified
SharedConsumerAssignorto accept aSubscriptionparameter for logging and to process unassigned messages instead of releasing them - Updated both
PersistentDispatcherMultipleConsumersandPersistentDispatcherMultipleConsumersClassicto pass the subscription to the assignor - Added logging when no consumer is found to assign messages
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| pulsar-broker/src/main/java/org/apache/pulsar/broker/service/SharedConsumerAssignor.java | Core fix: Added subscription parameter, logging, and changed to use unassignedMessageProcessor instead of releasing entries when no consumers are available |
| pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumers.java | Updated SharedConsumerAssignor constructor call to pass subscription parameter |
| pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentDispatcherMultipleConsumersClassic.java | Updated SharedConsumerAssignor constructor call to pass subscription parameter |
| pulsar-broker/src/test/java/org/apache/pulsar/broker/service/SharedConsumerAssignorTest.java | Updated test to pass null for subscription parameter in constructor |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/SharedConsumerAssignor.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/SharedConsumerAssignor.java
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/SharedConsumerAssignor.java
Outdated
Show resolved
Hide resolved
@lhotari Okay, I will make additional comments |
…t when no consumers were online
|
@lhotari I have submitted the test cases, please review them |
…t when no consumers were online
|
@lhotari Hello, please review it |
Motivation
When no consumers are online for a subscription, sending chunked messages will be lost.
Modifications
When no subscribers are online, send chunk messages to deliver them to the redeliveryMessages queue. When a subscriber comes online, it can consume the messages again.
Documentation
docdoc-requireddoc-not-neededdoc-complete