gh-124878: Fix race conditions during interpreter finalization#130649
Uh oh!
There was an error while loading. Please reload this page.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
The PyThreadState field gains a reference count field to avoid issues with PyThreadState being a dangling pointer to freed memory. The refcount starts with a value of two: one reference is owned by the interpreter's linked list of thread states and one reference is owned by the OS thread. The reference count is decremented when the thread state is removed from the interpreter's linked list and before the OS thread calls `PyThread_hang_thread()`. The thread that decrements it to zero frees the `PyThreadState` memory. The `holds_gil` field is moved out of the `_status` bit field, to avoid a data race where on thread calls `PyThreadState_Clear()`, modifying the `_status` bit field while the OS thread reads `holds_gil` when attempting to acquire the GIL. The `PyThreadState.state` field now has `_Py_THREAD_SHUTTING_DOWN` as a possible value. This corresponds to the `_PyThreadState_MustExit()` check. This avoids race conditions in the free threading build when checking `_PyThreadState_MustExit()`.
vstinner approved these changes Feb 28, 2025
Member
vstinner left a comment
vstinner left a comment There was a problem hiding this comment.
LGTM.
Avoiding dangling pointers close a whole category of bugs. Using a reference count is a nice solution for that. I also like the fact that Py_Finalize() now sets threads state to "shutting down": it's more explicit like that.
Member
vstinner commented Feb 28, 2025
By the way, this change may also fix the old crash #110052 (test_4_daemon_threads). |
ContributorAuthor
colesbury commented Feb 28, 2025
Yeah, I think it should fix
And also a small patch to ceval_gil.c to make the crash more likely to occur on my machine. On main (with the patch to ceval_gil.c), I see a crash pretty quickly after ~100 iterations on both the default and free threaded builds. With this PR, I haven't seen any failures in 25,000 iterations (~10 minutes). Patch to ceval_gil.cdiff --git a/Python/ceval_gil.c b/Python/ceval_gil.c index 2c1cc17b2ff..e14f1a8afa2 100644 --- a/Python/ceval_gil.c+++ b/Python/ceval_gil.c@@ -306,6 +306,8 @@ take_gil(PyThreadState *tstate) _PyThreadState_HangThread(tstate)} + usleep(10);+ assert(_PyThreadState_CheckConsistency(tstate)); PyInterpreterState *interp = tstate->interp; struct _gil_runtime_state *gil = interp->ceval.gil; |
colesbury merged commit 052cb71 into python:main 42 checks passed
Uh oh!
There was an error while loading. Please reload this page.
bedevere-bot commented Mar 6, 2025
|
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The PyThreadState field gains a reference count field to avoid issues with PyThreadState being a dangling pointer to freed memory. The refcount starts with a value of two: one reference is owned by the interpreter's linked list of thread states and one reference is owned by the OS thread. The reference count is decremented when the thread state is removed from the interpreter's linked list and before the OS thread calls
PyThread_hang_thread(). The thread that decrements it to zero frees thePyThreadStatememory.The
holds_gilfield is moved out of the_statusbit field, to avoid a data race where on thread callsPyThreadState_Clear(), modifying the_statusbit field while the OS thread readsholds_gilwhen attempting to acquire the GIL.The
PyThreadState.statefield now has_Py_THREAD_SHUTTING_DOWNas a possible value. This corresponds to the_PyThreadState_MustExit()check. This avoids race conditions in the free threading build when checking_PyThreadState_MustExit().