bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of freed memory#30434
bpo-46263: FreeBSD 14.0 jemalloc workaround for junk bytes of freed memory#30434miss-islington merged 1 commit intopython:mainfrom
Conversation
|
|
@tiran: Status check is done, and it's a success ✅ . |
|
Thanks @tiran for the PR 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10. |
…emory (pythonGH-30434) Automerge-Triggered-By: GH:tiran (cherry picked from commit a4aa52d) Co-authored-by: Christian Heimes <[email protected]>
|
GH-30436 is a backport of this pull request to the 3.10 branch. |
…emory (pythonGH-30434) Automerge-Triggered-By: GH:tiran (cherry picked from commit a4aa52d) Co-authored-by: Christian Heimes <[email protected]>
|
GH-30437 is a backport of this pull request to the 3.9 branch. |
…emory (GH-30434) Automerge-Triggered-By: GH:tiran (cherry picked from commit a4aa52d) Co-authored-by: Christian Heimes <[email protected]>
…reed memory (GH-30434) (GH-30437) Co-authored-by: Christian Heimes <[email protected]>
| out = assert_python_failure( | ||
| '-c', code, | ||
| PYTHONMALLOC=self.PYTHONMALLOC, | ||
| # FreeBSD: instruct jemalloc to not fill freed() memory |
There was a problem hiding this comment.
this is a rather fragile test as written. depending on the undefined behavior of what the contents of free()d memory will be is our problem. jemalloc is doing something perfectly reasonable, it's our test that is not. expect this to come up again at a random future point even if this workaround does something useful today.
There was a problem hiding this comment.
Other memory allocators may use different constants. I have an open PR for mimalloc microsoft/mimalloc#517
There was a problem hiding this comment.
Maybe _PyMem_IsPtrFreed() constants could be updated at runtime if Python detects when jemalloc is running?
I already had to change _PyMem_IsPtrFreed() constants when I implemented this function, to use Microsoft memory allocator constants.
_PyMem_IsPtrFreed() is only called by _PyObject_IsFreed() which is only used when Python is built in debug mode (and a few _testcapi test functions in release mode as well). So performance is not critical here.
There was a problem hiding this comment.
You would have to detect that you run under jemalloc and jemalloc is built with debug flags or jemalloc is configured to taint freed memory. FreeBSD 14.0 is under development and its libc uses a debug build. Release builds of FreeBSD are not affected.
There was a problem hiding this comment.
A lot of memory allocators do this when not building release/optimized builds. It isn't just jemalloc. I don't see an obvious clean solution.
7 participants
https://bugs.python.org/issue46263
Automerge-Triggered-By: GH:tiran