Starting new thread during finalization leads to call to `PyMem_Free` without holding the GIL

[chgnrdv]

Crash report

What happened?

Since #109135, thread_run checks if it is called during finalization, and in this case it frees thread bootstate in thread_bootstate_free and returns. The problem is that PyMem_Free shouldn't be called if GIL is not held.

Repro and error message (just in case):

import os
import random
import signal
import subprocess
import sys
import time

script = """
import threading
while True:
 t = threading.Thread()
 t.start()
 t.join()
"""

while True:
 p = subprocess.Popen([sys.executable, '-c', script], stderr=subprocess.PIPE)
 time.sleep(random.random())
 os.kill(p.pid, signal.SIGINT)
 _, err = p.communicate()
 if p.returncode == -signal.SIGABRT:
 print(err.decode('utf-8'))
 break

Traceback (most recent call last):
 File "<string>", line 5, in <module>
 File "/home/radislav/projects/cpython/Lib/threading.py", line 983, in start
 self._started.wait()
 File "/home/radislav/projects/cpython/Lib/threading.py", line 641, in wait
 signaled = self._cond.wait(timeout)
 ^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/radislav/projects/cpython/Lib/threading.py", line 341, in wait
 waiter.acquire()
KeyboardInterrupt
Fatal Python error: _PyMem_DebugFree: Python memory allocator called without holding the GIL
Python runtime state: finalizing (tstate=0x0000561b61b67338)

Thread 0x00007f8f890b8280 (most recent call first):
 <no Python frame>

cc @vstinner

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Output from running 'python -VV' on the command line:

Python 3.13.0a0 (heads/main:3e8fcb7df7, Sep 23 2023, 01:43:45) [GCC 10.2.1 20210110]

Linked PRs

• gh-109795: _thread.start_new_thread: allocate thread bootstate using raw memory allocator #109808
• [3.11] gh-109795: _thread.start_new_thread: allocate thread bootstate usin… #109852

[vstinner]

thread_PyThread_start_new_thread() can easily use PyMem_RawMalloc() instead of PyMem_Malloc(). Do you want to propose a fix?

[chgnrdv]

@vstinner, yes, I'll submit a PR shortly :)

[vstinner]

Ah. After many attempts, I managed to reproduce the bug without the fix:

$ ./python repro.py 
...............failed with exit code -6
Fatal Python error: _PyMem_DebugFree: Python memory allocator called without holding the GIL
Python runtime state: finalizing (tstate=0x0000000000b062f8)

Thread 0x00007fd52873a740 (most recent call first):
 <no Python frame>

I modified the reproducer script:

import os
import random
import signal
import subprocess
import sys
import time

script = """
try:
 import threading
 while True:
 t = threading.Thread()
 t.start()
 t.join()
except KeyboardInterrupt:
 pass
"""

line = 0

while True:
 if line % 40 == 0:
 sys.stdout.write(f"\n{line} ")
 sys.stdout.write(".")
 sys.stdout.flush()
 line += 1

 p = subprocess.Popen([sys.executable, '-c', script], stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
 time.sleep(0.10 + random.random()*0.5)
 os.kill(p.pid, signal.SIGINT)
 with p:
 try:
 out = p.communicate(timeout=1.0)[0]
 except subprocess.TimeoutExpired:
 sys.stdout.write("*** timeout ***")
 sys.stdout.flush()
 p.kill()
 continue
 if p.returncode:
 print(f"failed with exit code {p.returncode}")
 print(out.decode('utf-8'))
 break

Sometimes, I get other bugs like:

$ ./python repro.py 

0 ...................................failed with exit code 1
Traceback (most recent call last):
 File "/home/vstinner/python/main/Lib/threading.py", line 641, in wait
 signaled = self._cond.wait(timeout)
 ^^^^^^^^^^^^^^^^^^^^^^^^
 File "/home/vstinner/python/main/Lib/threading.py", line 337, in wait
 saved_state = self._release_save()
 ^^^^^^^^^^^^^^^^^^^^
 File "/home/vstinner/python/main/Lib/threading.py", line 295, in _release_save
 self._lock.release() # No state to save
 ^^^^^^^^^^^^^^^^^^^^
KeyboardInterrupt

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "<string>", line 6, in <module>
 File "/home/vstinner/python/main/Lib/threading.py", line 983, in start
 self._started.wait()
 File "/home/vstinner/python/main/Lib/threading.py", line 638, in wait
 with self._cond:
 File "/home/vstinner/python/main/Lib/threading.py", line 289, in __exit__
 return self._lock.__exit__(*args)
 ^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: release unlocked lock

[vstinner]

Thanks for the reproducer @chgnrdv!

The bug is hard to trigger :-( But apparently, I cannot reproduce the bug anymore. Good :-)

[chgnrdv]

@vstinner thank you for review :)

Yep, it takes precise timing for interrupt to make Python finalize before new thread will be started in thread_PyThread_start_new_thread. The repro I attached is fittable for my machine, but may not work on others.