libn/networkdb: don't exceed broadcast size limit #49939
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
NetworkDB uses a hierarchy of queues to prioritize messages for broadcast. Unfortunately the logic to pull from multiple queues is flawed. The length of the messages pulled from the first queue is not taken into account when pulling messages from the second queue. A list of messages up to tiwce as long as the limit could be returned! Messages beyond the limit will be truncated unceremoniously by memberlist. Memberlist broadcast queues assume that all messages returned from a GetBroadcasts call will be broadcasted to other nodes in the cluster. Messages are popped from the queue once they have hit their retransmit limit. On a busy system messages may be broadcast fewer times than intended, possibly even being dropped without ever being broadcast! Subtract the length of messages pulled from the first queue from the broadcast size limit so the limit is not exceeded when pulling from the second queue. Signed-off-by: Cory Snider <[email protected]>
corhere
added
area/networking
impact/changelog
process/cherry-pick
kind/bugfix
corhere
force-pushed
the
libn/networkdb-broadcast-overflow
branch
from
3185bb6 to
dacf445
Compare
robmry
approved these changes
May 8, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
- What I did
NetworkDB uses a hierarchy of queues to prioritize messages for broadcast. Unfortunately the logic to pull from multiple queues is flawed. The length of the messages pulled from the first queue is not taken into account when pulling messages from the second queue. A list of messages up to tiwce as long as the limit could be returned! Messages beyond the limit will be truncated unceremoniously by memberlist.
Memberlist broadcast queues assume that all messages returned from a GetBroadcasts call will be broadcasted to other nodes in the cluster. Messages are popped from the queue once they have hit their retransmit limit. On a busy system messages may be broadcast fewer times than intended, possibly even being dropped without ever being broadcast!
I fixed this oversight so that the delegate never returns more data than the specified length limit.
- How I did it
Subtract the length of messages pulled from the first queue from the broadcast size limit so the limit is not exceeded when pulling from the second queue.
- How to verify it
By inspection 🤷
- Human readable description for the release notes
- A picture of a cute animal (not mandatory but encouraged)