cProfile hangs / segfaults with 3.13t

[andrewkho]

Bug report

Bug description:

I'm testing Python 3.13t and trying to run cProfile in a context manager, however it seems to hang and/or segfault.

Below script repros the issue.

With > python -Xgil=1 test_cProfile.py profile
it succeeds as expected, however disabling -Xgil=0 causes it to usually hang.

""" example script showing deadlock with python 3.13t and cProfile context manager
python -Xgil=0 test_cProfile.py # runs normally
python -Xgil=0 test_cProfile.py profile # hangs
"""
import cProfile
import sys
import time
import threading
import queue
def worker(wid, q, stop):
 while not stop.is_set():
 try:
 x = q.get(timeout=1.)
 print(wid, x)
 except queue.Empty:
 continue
def run():
 stop = threading.Event()
 workers = []
 queues = []
 num_workers=2
 for wid in range(num_workers):
 queues.append(queue.Queue())
 workers.append(threading.Thread(target=worker, args=(wid, queues[-1], stop)))
 workers[-1].start()
 
 for i in range(100):
 queues[i%num_workers].put(i)
 while any(q.qsize() for q in queues):
 print("Waiting...")
 time.sleep(0.1)
 continue
 print("stop.set()")
 stop.set()
 for wid in range(num_workers):
 print("Joining", wid)
 workers[wid].join()
 
 print("Done")
def main():
 if len(sys.argv) > 1 and sys.argv[1] == "profile":
 print("Profiling with cProfile")
 with cProfile.Profile() as pr:
 run()
 else:
 print("Running without cProfile")
 run()
if __name__ == "__main__":
 main()
 commented on this paste.```


### CPython versions tested on:

3.13

### Operating systems tested on:

Linux

[colesbury]

Modules/_lsprof.c is not thread-safe without the GIL. I think per-profiler locks won't be efficient and would limit the usefulness for profiling multithreaded programs in the free threading builds.

As a first step, I think we should probably:

• Remove the Py_MOD_GIL_NOT_USED from _lsprof.c
• Document it as a known limitation in https://docs.python.org/3.13/howto/free-threading-python.html#freethreading-python-howto

Then we can figure out how to make it thread-safe and efficient.

[Zheaoli]

I would like to follow up this issue if there is no one working on it.

[kumaraditya303]

Fixed by #138229

Output on current main:

Profiling with cProfile
0 0
1 1
0 2
1 3
0 4
1 5
0 6
1 7
0 8
1 9
0 10
1 11
0 12
1 13
0 14
1 15
0 16
1 17
0 18
1 19
0 20
1 21
0 22
1 23
0 24
1 25
0 26
1 27
0 28
1 29
0 30
1 31
0 32
1 33
0 34
1 35
0 36
1 37
0 38
1 39
0 40
1 41
0 42
1 43
0 44
1 45
0 46
1 47
0 48
1 49
0 50
1 51
0 52
1 53
0 54
1 55
0 56
1 57
0 58
1 59
0 60
1 61
0 62
1 63
0 64
1 65
0 66
1 67
0 68
1 69
0 70
1 71
0 72
1 73
0 74
1 75
0 76
1 77
0 78
1 79
0 80
1 81
0 82
1 83
0 84
1 85
0 86
1 87
0 88
1 89
0 90
1 91
0 92
1 93
0 94
1 95
0 96
1 97
0 98
1 99
stop.set()
Joining 0
Joining 1
Done