Skip to content

bpo-30966: concurrent.futures.Process.shutdown() closes queue#19738

Merged
vstinner merged 1 commit intopython:masterfrom
vstinner:concurrent_queue_close
Apr 27, 2020
Merged

bpo-30966: concurrent.futures.Process.shutdown() closes queue#19738
vstinner merged 1 commit intopython:masterfrom
vstinner:concurrent_queue_close

Conversation

@vstinner
Copy link
Member

@vstinner vstinner commented Apr 27, 2020 •
edited by bedevere-bot
Loading

Process.shutdown(wait=True) of concurrent.futures now closes
explicitly the result queue.

https://bugs.python.org/issue30966

@vstinner
bpo-30966: concurrent.futures.Process.shutdown() closes queue
9333caa 
Process.shutdown(wait=True) of concurrent.futures now closes
explicitly the result queue.
@vstinner vstinner merged commit 1a27501 into python:master Apr 27, 2020
@bedevere-bot
Copy link

bedevere-bot commented Apr 27, 2020

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Ubuntu Shared 3.x has failed when building commit 1a27501.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/101/builds/809) and take a look at the build logs.
  4. Check if the failure is related to this commit (1a27501) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/101/builds/809

Failed tests:

  • test_concurrent_futures

Failed subtests:

  • test_shutdown_no_wait - test.test_concurrent_futures.ProcessPoolForkserverProcessPoolShutdownTest
  • test_shutdown_no_wait - test.test_concurrent_futures.ProcessPoolForkProcessPoolShutdownTest

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

405 tests OK.

10 slowest tests:

  • test_multiprocessing_spawn: 5 min 3 sec
  • test_concurrent_futures: 4 min 38 sec
  • test_peg_generator: 3 min 31 sec
  • test_unparse: 2 min 54 sec
  • test_tokenize: 2 min 51 sec
  • test_capi: 2 min 45 sec
  • test_gdb: 2 min 23 sec
  • test_asyncio: 2 min 22 sec
  • test_multiprocessing_forkserver: 2 min 21 sec
  • test_lib2to3: 2 min 17 sec

1 test failed:
test_concurrent_futures

17 tests skipped:
test_devpoll test_idle test_ioctl test_kqueue test_msilib
test_ossaudiodev test_startfile test_tcl test_tix test_tk
test_ttk_guionly test_ttk_textonly test_turtle test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_concurrent_futures

Total duration: 35 min 50 sec

Click to see traceback logs 
Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_concurrent_futures.py", line 542, in test_shutdown_no_wait
    executor.shutdown(wait=False)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/concurrent/futures/process.py", line 724, in shutdown
    self._executor_manager_thread_wakeup.wakeup()
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/concurrent/futures/process.py", line 80, in wakeup
    self._writer.send_bytes(b"")
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/connection.py", line 205, in send_bytes
    self._send_bytes(m[offset:offset + size])
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/connection.py", line 416, in _send_bytes
    self._send(header + buf)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/connection.py", line 373, in _send
    n = write(self._handle, buf)
OSError: [Errno 9] Bad file descriptor


Traceback (most recent call last):
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/test/test_concurrent_futures.py", line 542, in test_shutdown_no_wait
    executor.shutdown(wait=False)
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/concurrent/futures/process.py", line 724, in shutdown
    self._executor_manager_thread_wakeup.wakeup()
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/concurrent/futures/process.py", line 80, in wakeup
    self._writer.send_bytes(b"")
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/connection.py", line 188, in send_bytes
    self._check_closed()
  File "/srv/buildbot/buildarea/3.x.bolen-ubuntu/build/Lib/multiprocessing/connection.py", line 141, in _check_closed
    raise OSError("handle is closed")
OSError: handle is closed

@vstinner vstinner deleted the concurrent_queue_close branch April 27, 2020 19:46
@vstinner
Copy link
Member Author

vstinner commented Apr 27, 2020

Failed subtests:
test_shutdown_no_wait - test.test_concurrent_futures.ProcessPoolForkserverProcessPoolShutdownTest
test_shutdown_no_wait - test.test_concurrent_futures.ProcessPoolForkProcessPoolShutdownTest

This is https://bugs.python.org/issue39995

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vstinner @bedevere-bot @the-knights-who-say-ni