Skip to content

bpo-44422: Fix threading.enumerate() reentrant call#26727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vstinner merged 3 commits into from
Jun 15, 2021
Merged

bpo-44422: Fix threading.enumerate() reentrant call #26727

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
28fa7d7
bpo-44422: Fix threading.enumerate() reentrant call
vstinner Jun 14, 2021
adb0a4d
Fix typo
vstinner Jun 15, 2021
20a0d9a
Fix _after_fork()
vstinner Jun 15, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 6 additions & 3 deletions Lib/threading.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -775,8 +775,11 @@ class BrokenBarrierError(RuntimeError):
def _newname(name_template):
return name_template % _counter()

# Active thread administration
_active_limbo_lock = _allocate_lock()
# Active thread administration.
#
# bpo-44422: Use a reentrant lock to allow reentrant calls to functions like
# threading.enumerate().
_active_limbo_lock = RLock()
Copy link
Contributor

@rhettingerrhettingerJun 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The RLock() is the right way to go to prevent self-deadlock.

We could temporarily turn GC off but that seems hard to do in a way that reliably restores the prior enabled/disabled mode.

Copy link
MemberAuthor

@vstinnervstinnerJun 14, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could temporarily turn GC off but that seems hard to do in a way that reliably restores the prior enabled/disabled mode.

Sure, that's the other option that I considered. But gc.disable() is process-wide: it affects all Python threads, and so it might have surprising side effects. Some code might rely on the current exact GC behavior.

Another option would be to rewrite the code in C. The problem is that other functions rely on this lock, like active_count(). I would prefer to not have to rewrite "half" of threading.py in C. Using a RLock is less intrusive.

_active ={} # maps thread id to Thread object
_limbo ={}
_dangling = WeakSet()
Expand DownExpand Up@@ -1564,7 +1567,7 @@ def _after_fork():
# by another (non-forked) thread. http://bugs.python.org/issue874900
global _active_limbo_lock, _main_thread
global _shutdown_locks_lock, _shutdown_locks
_active_limbo_lock = _allocate_lock()
_active_limbo_lock = RLock()

# fork() only copied the current thread; clear references to others.
new_active ={}
Expand Down
3 changes: 3 additions & 0 deletions Misc/NEWS.d/next/Library/2021-06-14-23-28-17.bpo-44422.BlWOgv.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
The :func:`threading.enumerate` function now uses a reentrant lock to
prevent a hang on reentrant call.
Patch by Victor Stinner.