gh-137400: Fix a crash when disabling profiling across all threads

[colesbury]

The PyEval_SetProfileAllThreads function and other related functions had a race condition on tstate->c_profilefunc that could lead to a crash when disable profiling or tracing on all threads while another thread is starting to profile or trace a a call.

There are still potential crashes when threads exit concurrently with profiling or tracing be enabled/disabled across all threads.

• Issue: sys._setprofileallthreads race condition #137400

[colesbury]

Closing in favor of the more complete PR: #137518

[bedevere-bot]

🤖 New build scheduled with the buildbot fleet by @colesbury for commit 5e5e4f1 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F137471%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

[miss-islington-app]

Thanks @colesbury for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14.
🐍🍒⛏🤖

[miss-islington-app]

Sorry, @colesbury, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 362692852f13cdd1d33cc7ed35c0cbac7af1a785 3.13 

[bedevere-app]

GH-137648 is a backport of this pull request to the 3.14 branch.

[bedevere-app]

GH-137649 is a backport of this pull request to the 3.13 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Android 3.14 (tier-3) has failed when building commit 4ebd928.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1794/builds/435) and take a look at the build logs.
4. Check if the failure is related to this commit (4ebd928) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1794/builds/435

Summary of the results of the build (if available):

Click to see traceback logs

remote: Enumerating objects: 24, done. remote: Counting objects: 4% (1/24) remote: Counting objects: 8% (2/24) remote: Counting objects: 12% (3/24) remote: Counting objects: 16% (4/24) remote: Counting objects: 20% (5/24) remote: Counting objects: 25% (6/24) remote: Counting objects: 29% (7/24) remote: Counting objects: 33% (8/24) remote: Counting objects: 37% (9/24) remote: Counting objects: 41% (10/24) remote: Counting objects: 45% (11/24) remote: Counting objects: 50% (12/24) remote: Counting objects: 54% (13/24) remote: Counting objects: 58% (14/24) remote: Counting objects: 62% (15/24) remote: Counting objects: 66% (16/24) remote: Counting objects: 70% (17/24) remote: Counting objects: 75% (18/24) remote: Counting objects: 79% (19/24) remote: Counting objects: 83% (20/24) remote: Counting objects: 87% (21/24) remote: Counting objects: 91% (22/24) remote: Counting objects: 95% (23/24) remote: Counting objects: 100% (24/24) remote: Counting objects: 100% (24/24), done. remote: Compressing objects: 7% (1/13) remote: Compressing objects: 15% (2/13) remote: Compressing objects: 23% (3/13) remote: Compressing objects: 30% (4/13) remote: Compressing objects: 38% (5/13) remote: Compressing objects: 46% (6/13) remote: Compressing objects: 53% (7/13) remote: Compressing objects: 61% (8/13) remote: Compressing objects: 69% (9/13) remote: Compressing objects: 76% (10/13) remote: Compressing objects: 84% (11/13) remote: Compressing objects: 92% (12/13) remote: Compressing objects: 100% (13/13) remote: Compressing objects: 100% (13/13), done. remote: Total 13 (delta 11), reused 0 (delta 0), pack-reused 0 (from 0)  From https://github.com/python/cpython * branch 3.14 -> FETCH_HEAD Note: switching to '4ebd928b0096ca1dd78d5d3e6c787dbc40a43c11'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c <new-branch-name> Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 4ebd928b009 [3.14] gh-137400: Fix a crash when disabling profiling across all threads (gh-137471) (#137648) Switched to and reset branch '3.14' configure: WARNING: no system libmpdec found; falling back to bundled libmpdec (deprecated and scheduled for removal in Python 3.16)configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 192k 100 192k 0 0 1474k 0 --:--:-- --:--:-- --:--:-- 1474k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 42455 100 42455 0 0 318k 0 --:--:-- --:--:-- --:--:-- 318k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 6 5041k 6 333k 0 0 2019k 0 0:00:02 --:--:-- 0:00:02 2019k 100 5041k 100 5041k 0 0 24.6M 0 --:--:-- --:--:-- --:--:-- 135M % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 1257k 100 1257k 0 0 9634k 0 --:--:-- --:--:-- --:--:-- 9634k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 635k 100 635k 0 0 5066k 0 --:--:-- --:--:-- --:--:-- 5066k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 479k 100 479k 0 0 2199k 0 --:--:-- --:--:-- --:--:-- 2199k ../../configure: line 4071: pkg-config: command not foundconfigure: WARNING: no system libmpdec found; falling back to bundled libmpdec (deprecated and scheduled for removal in Python 3.16)configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly. ../../Python/fileutils.c:460:1: warning: unused function 'decode_current_locale' [-Wunused-function] 460 | decode_current_locale(const char* arg, wchar_t **wstr, size_t *wlen, |^~~~~~~~~~~~~~~~~~~~~ ../../Python/fileutils.c:679:1: warning: unused function 'encode_current_locale' [-Wunused-function] 679 | encode_current_locale(const wchar_t *text, char **str, |^~~~~~~~~~~~~~~~~~~~~ 2 warnings generated. ../../Modules/_localemodule.c:148:1: warning: unused function 'is_all_ascii' [-Wunused-function] 148 | is_all_ascii(const char *str) |^~~~~~~~~~~~ 1 warning generated. ../../Modules/_hacl/Lib_Memzero0.c:66:6: warning: "Your platform does not support any safe implementation of memzero -- consider a pull request!" [-W#warnings] 66 | #warning "Your platform does not support any safe implementation of memzero -- consider a pull request!" |^ 1 warning generated. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 8784 100 8784 0 0 67630 0 --:--:-- --:--:-- --:--:-- 68093 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 2894 100 2894 0 0 9811 0 --:--:-- --:--:-- --:--:-- 9843 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 43504 100 43504 0 0 696k 0 --:--:-- --:--:-- --:--:-- 708k

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Android 3.14 (tier-3) has failed when building commit 4ebd928.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1794/builds/436) and take a look at the build logs.
4. Check if the failure is related to this commit (4ebd928) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1794/builds/436

Summary of the results of the build (if available):

Click to see traceback logs

Note: switching to '4ebd928b0096ca1dd78d5d3e6c787dbc40a43c11'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c <new-branch-name> Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 4ebd928b009 [3.14] gh-137400: Fix a crash when disabling profiling across all threads (gh-137471) (#137648) Switched to and reset branch '3.14' configure: WARNING: no system libmpdec found; falling back to bundled libmpdec (deprecated and scheduled for removal in Python 3.16)configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 192k 100 192k 0 0 1201k 0 --:--:-- --:--:-- --:--:-- 1201k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 42455 100 42455 0 0 419k 0 --:--:-- --:--:-- --:--:-- 419k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 5041k 100 5041k 0 0 30.7M 0 --:--:-- --:--:-- --:--:-- 30.7M % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 1257k 100 1257k 0 0 11.6M 0 --:--:-- --:--:-- --:--:-- 11.6M % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 635k 100 635k 0 0 4794k 0 --:--:-- --:--:-- --:--:-- 4794k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 479k 100 479k 0 0 4090k 0 --:--:-- --:--:-- --:--:-- 4090k ../../configure: line 4071: pkg-config: command not foundconfigure: WARNING: no system libmpdec found; falling back to bundled libmpdec (deprecated and scheduled for removal in Python 3.16)configure: WARNING: pkg-config is missing. Some dependencies may not be detected correctly. ../../Python/fileutils.c:460:1: warning: unused function 'decode_current_locale' [-Wunused-function] 460 | decode_current_locale(const char* arg, wchar_t **wstr, size_t *wlen, |^~~~~~~~~~~~~~~~~~~~~ ../../Python/fileutils.c:679:1: warning: unused function 'encode_current_locale' [-Wunused-function] 679 | encode_current_locale(const wchar_t *text, char **str, |^~~~~~~~~~~~~~~~~~~~~ 2 warnings generated. ../../Modules/_localemodule.c:148:1: warning: unused function 'is_all_ascii' [-Wunused-function] 148 | is_all_ascii(const char *str) |^~~~~~~~~~~~ 1 warning generated. ../../Modules/_hacl/Lib_Memzero0.c:66:6: warning: "Your platform does not support any safe implementation of memzero -- consider a pull request!" [-W#warnings] 66 | #warning "Your platform does not support any safe implementation of memzero -- consider a pull request!" |^ 1 warning generated. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 8784 100 8784 0 0 142k 0 --:--:-- --:--:-- --:--:-- 142k % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 2894 100 2894 0 0 55255 0 --:--:-- --:--:-- --:--:-- 55653 % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 43504 100 43504 0 0 926k 0 --:--:-- --:--:-- --:--:-- 944k

[hugovk]

(These were temporary failures, fixed by a buildbot restart: #137186 (comment))