The Wayback Machine - https://web.archive.org/web/20250120183904/https://github.com/apache/thrift/pull/2496
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

THRIFT-5492: Add readEnd to TBufferedTransport #2496

Closed
wants to merge 1 commit into from

Conversation

slicking
Copy link
Contributor

client: cpp

Add a readEnd implementation to TBufferedTransport which resets the consumed
message size. Without this it was noticed that calls to the transport's
consume method would slowly decrement the remainingMessageSize_ and the
remainingMessageSize_ was never reset. A client could send many messages to
the server, eventually bringing remainingMessageSize_ below zero causing an
EOD_OF_FILE exception.

client: cpp

Add a readEnd implementation to TBufferedTransport which resets the consumed
message size.  Without this it was noticed that calls to the transport's
consume method would slowly decrement the remainingMessageSize_ and the
remainingMessageSize_ was never reset.  A client could send many messages to
the server, eventually bringing remainingMessageSize_ below zero causing an
EOD_OF_FILE exception.
@Jens-G
Copy link
Member

Jens-G commented Jan 8, 2022

This stage failed.

Job Xcode ENV OS State
7919.3   SCRIPT="cmake.sh" Linux failed
7919.4   SCRIPT="cmake.sh" Linux failed
7919.5   SCRIPT="cmake.sh" Linux failed
7919.6   SCRIPT="cmake.sh" Linux failed
7919.7   SCRIPT="cmake.sh" Linux failed
7919.8   SCRIPT="cmake.sh" Linux failed
7919.9   SCRIPT="cmake.sh" Linux failed
7919.10   SCRIPT="cmake.sh" Linux failed
7919.11   SCRIPT="cmake.sh" Linux passed
7919.12   SCRIPT="cmake.sh" Linux failed
7919.13   SCRIPT="cmake.sh" Linux passed
7919.14 xcode11.3 SCRIPT="cmake.sh" macOS passed

@slicking
Copy link
Contributor Author

@Jens-G thanks for pointing out the failures. Looking at several of them it appears to be because I added to test/StessTest.thrift but only updated test/cpp/src/StessTest.cpp and NOT lib/d/test/stress_test_server.d so there is an unimplemented function.
The fix I made was only under the c++ server; let me see if adding a test under lib/cpp/test/ would be better contained and cover only the c++ server.

@Jens-G Jens-G added the c++ label Oct 25, 2022
@emmenlau
Copy link
Member

This looks promising and relevant. @slicking , there is a small favor that would be very helpful in moving this forward. Could you provide a separate PR with a test case, that shows that indeed client messages would bring remainingMessageSize_ to zero (or below), thereby causing an EOD_OF_FILE exception? If this can be demonstrated in a small test, it would point to a severe problem in current thrift, and nicely motivate this PR.

Do you think you could provide a test?

@goorjn
Copy link

goorjn commented Jun 27, 2024

Do you think Message Size check is implemented the right way ?
Is this not a protocol concern, rather than a distributed protocol / transport responsibility ?
The functionality is implemented through transport base class function definition. Don't you think it would be better to move these to a different object which responsibility would be to check message size validity ?
Several functions in the implementation are never called (in production code, I haven't checked the tests), code is hard to read and some paths are never reached (resetConsumedMessageSize(long newSize = -1) with actual newSize value).
I do not know enough about the whole Virtual protocol / transport implementation to propose relevant changes to the code, but the current one (I use version 0.14.3) broke my program !

@jsiewkow
Copy link

jsiewkow commented Jul 9, 2024

I've moved from 0.11.0 to 0.20.0 and encountered the issue described here, fix in this PR seems to solve it, is there a plan for this change to be merged?

@Jens-G
Copy link
Member

Jens-G commented Jul 9, 2024

I merged the fix part in 86b05bf, but left the rest, as there are unfortunately still open questions etc.

@jsiewkow
Copy link

Thank you, I guess this will be in next thrift release, when do you plan it?

@emmenlau
Copy link
Member

emmenlau commented Aug 7, 2024

Is the remainder of this PR still valid? Anyone able to comment on the remaining issues?

@slicking
Copy link
Contributor Author

slicking commented Aug 7, 2024

Thanks @Jens-G for merging the fix; I'm glad it could be of help to a few others.
I've not had time to get back to this issue and update the test; with the fix merged though I think this PR can be closed. The changes to update the test in this PR need to be reworked to be specific to the c++ server and, if anyone is able, can be taken up in a separate PR.

@slicking slicking closed this Aug 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants