Message401146
Ok, that's what I was able to find.
Original BPO: https://bugs.python.org/issue35813
Original PRs (sources + tests):
- https://github.com/python/cpython/pull/11664 it was a little bit "rushed"
- https://github.com/python/cpython/pull/11816 follow up
Related: https://bugs.python.org/issue38169
1. The test fails because `sms.size != sms2.size`
2. The `sms2` object is a pickle-unpickle copy of `sms`
3. The `__reduce__` method defined in `SharedMemory` returns `(name=self.name, create=False, size=self.size)` tuple
4. So, when `sms2` is constructed, `size` is ignored, because `create` is `False`
Moreover, even when `size` is passed with `create=True` the docs say that the final amount of allocated memory is platform specific (no details on the exact platforms): it can be larger or equal. Link: https://github.com/python/cpython/blame/main/Doc/library/multiprocessing.shared_memory.rst#L61
So, this statement makes `self.assertEqual(sms.size, sms2.size)` potentially flaky.
After reading all this my suggestion is:
1. Remove `self.assertEqual(sms.size, sms2.size)` line, since this behavior is not guaranteed to work
2. Add docs about how `pickle` works with `SharedMemory` / `ShareableList`. Right now it is not clear. This should be just an explanation of the current status-quo
3. Add more tests on unpickled `Shared*` objects. Including overflowing set memory limits on unpickled objects and other implemetation details
I will start with `1.` right now, later I can work on `3.`.
But, I feel like someone more knowledgeable should take `2.` (or at least guide me, since `SharedMemory` is not something I use everyday). |
|
| Date |
User |
Action |
Args |
| 2021-09-06 15:49:50 | sobolevn | set | recipients:
+ sobolevn, paul.moore, pitrou, tim.golden, zach.ware, serhiy.storchaka, eryksun, steve.dower, davin |
| 2021-09-06 15:49:50 | sobolevn | set | messageid: <[email protected]> |
| 2021-09-06 15:49:50 | sobolevn | link | issue45052 messages |
| 2021-09-06 15:49:50 | sobolevn | create | |
|