fluentd: add write timeout log option #49911
Merged
+149
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
austinvazquez
approved these changes
May 3, 2025
aaithal
force-pushed
the
fluentd-write-timeout
branch
2 times, most recently
from
c129735 to
fc79f13
Compare
austinvazquez
added
area/logging
kind/enhancement
aaithal
force-pushed
the
fluentd-write-timeout
branch
from
fc79f13 to
880de67
Compare
thaJeztah
requested changes
May 9, 2025
| assert.NilError(t, err, "unable to create listener for socket %s", socketFile) | ||
| defer l.Close() | ||
|
|
||
| ctx, cancel := context.WithCancel(context.Background()) |
There was a problem hiding this comment.
I was trying the test with the option disabled, and that caused it (expectedly) to hang forever; perhaps we should add a timeout here .. just in case?
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
daemon/logger/fluentd/fluentd.go
Outdated
Comment on lines
249
to
250
| return config, errors.Wrapf(err, "invalid value for %s: value must be a duration", writeTimeoutKey) | ||
|
|
|
Thanks for reviewing this and for the comments. I can address a bulk of these and post an update. |
|
Thanks! Yes, give me a ping when you had time to do so; when doing so, please amend the commit (squashed); no need to keep them as separate commits ❤️ |
aaithal
force-pushed
the
fluentd-write-timeout
branch
from
880de67 to
18b8834
Compare
thaJeztah
reviewed
May 9, 2025
thaJeztah
reviewed
May 9, 2025
aaithal
force-pushed
the
fluentd-write-timeout
branch
from
18b8834 to
db52f6b
Compare
Currently, there's no mechanism to specify a write timeout value for fluentd connections. This means that writes can forever be blocked if the downstream connections is unhealthy. This commit makes this value configurable via a new fluentd log option called "fluentd-write-timeout". Signed-off-by: Anirudh Aithal <[email protected]>
aaithal
force-pushed
the
fluentd-write-timeout
branch
from
db52f6b to
7dae7c5
Compare
|
I think I've addressed all of the comments with this change: https://github.com/moby/moby/compare/db52f6b42724419a5f2140c740d5c87eae8f4534..7dae7c54dde81951330170c7678deea350b1d2b8. Can you please take another look @thaJeztah ? Thanks in advance! |
This was referenced May 15, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added support for
fluentd-write-timeoutconfiguration option in the fluentd logger to prevent indefinite blocking when downstream connections become unhealthy. This allows fluentd to abandon unresponsive connections after a specified duration instead of being blocked forever.Key changes:
writeTimeoutKeyconfiguration option- What I did
Modified
fluentdlogdriver code to expose a new log option. The log option can be used to specify write timeouts.- How I did it
The code change exposes the existing config to specify a write timeout value for writes to fluentd server.
- How to verify it
Added a unit test, which fails when the timeout is not specified. The test succeeds when a timeout value is specified.
- Human readable description for the release notes