bpo-38501: Add a warning section to multiprocessing.Pool docs about resource managing#19466
bpo-38501: Add a warning section to multiprocessing.Pool docs about resource managing#19466pablogsal merged 1 commit intopython:masterfrom
Conversation
|
|
||
| Note that is **not correct** to rely on the garbage colletor to destroy the pool | ||
| as CPython does not assure that the finalizer of the pool will be called | ||
| (see :meth:`object.__del__` for more information). |
There was a problem hiding this comment.
My potential fix is to use atexit.register when we can't use the context manager. Worth suggesting here? (Also let me know if it's clearly a bad idea for some reason!)
There was a problem hiding this comment.
My potential fix is to use
atexit.registerwhen we can't use the context manager. Worth suggesting here? (Also let me know if it's clearly a bad idea for some reason!)
That works, but the problem is that is still a lossy method of resource managing (you would not use that method for all your file objects for instance). Technically, someone that is not "the interpreter" should own the pool and should be in charge of destroying it correctly.
I understand that some architecture and designs make this challenging so things like atexit do work for those cases, but I think is better not to explicitly mention that here so users don't start to use that as a silver bullet and use more the context manager for instance.
What do you think?
There was a problem hiding this comment.
Yeah that makes sense. Otherwise, text LGTM and reads nicely in the rendered artifact!
|
Thanks @pablogsal for the PR 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8. |
…esource managing (pythonGH-19466) (cherry picked from commit 7ec43a7) Co-authored-by: Pablo Galindo <[email protected]>
|
GH-19467 is a backport of this pull request to the 3.8 branch. |
…esource managing (pythonGH-19466) (cherry picked from commit 7ec43a7) Co-authored-by: Pablo Galindo <[email protected]>
|
GH-19468 is a backport of this pull request to the 3.7 branch. |
…esource managing (GH-19466) (cherry picked from commit 7ec43a7) Co-authored-by: Pablo Galindo <[email protected]>
…esource managing (GH-19466) (cherry picked from commit 7ec43a7) Co-authored-by: Pablo Galindo <[email protected]>
5 participants
https://bugs.python.org/issue38501