gh-127989: C API: Refer to attached thread states instead of the GIL

Doc/c-api/dict.rst

@@ -127,7 +127,7 @@ Dictionary Objects
  Prefer the :c:func:`PyDict_GetItemWithError` function instead.
 
  .. versionchanged:: 3.10
- Calling this API without :term:`GIL` held had been allowed for historical
+ Calling this API without an :term:`attached thread state` had been allowed for historical
  reason. It is no longer allowed.
 
 

Doc/c-api/exceptions.rst

@@ -413,7 +413,7 @@ Querying the error indicator
  own a reference to the return value, so you do not need to :c:func:`Py_DECREF`
  it.
 
- The caller must hold the GIL.
+ The caller must have an :term:`attached thread state`.
 
  .. note::
 
@@ -675,7 +675,7 @@ Signal Handling
 
  .. note::
  This function is async-signal-safe. It can be called without
-the :term:`GIL` and from a C signal handler.
+an :term:`attached thread state` and from a C signal handler.
 
 
 .. c:function:: int PyErr_SetInterruptEx(int signum)
@@ -702,7 +702,7 @@ Signal Handling
 
  .. note::
  This function is async-signal-safe. It can be called without
-the :term:`GIL` and from a C signal handler.
+an :term:`attached thread state` and from a C signal handler.
 
  .. versionadded:: 3.10
 

Doc/c-api/init_config.rst

@@ -578,8 +578,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option
  * ``list[str]``
  * ``dict[str, str]``
 
- The caller must hold the GIL. The function cannot be called before
- Python initialization nor after Python finalization.
+ The caller must have an :term:`attached thread state`. The function cannot
+be called before Python initialization nor after Python finalization.
 
  .. versionadded:: 3.14
 
@@ -601,8 +601,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option
  * Return a new reference on success.
  * Set an exception and return ``NULL`` on error.
 
- The caller must hold the GIL. The function cannot be called before
- Python initialization nor after Python finalization.
+ The caller must have an :term:`attached thread state`. The function cannot
+be called before Python initialization nor after Python finalization.
 
  .. versionadded:: 3.14
 
@@ -616,8 +616,8 @@ Some options are read from the :mod:`sys` attributes. For example, the option
  * Raise a :exc:`ValueError` if the option is read-only (cannot be set).
  * Raise a :exc:`TypeError` if *value* has not the proper type.
 
- The caller must hold the GIL. The function cannot be called before
- Python initialization nor after Python finalization.
+ The caller must have an :term:`attached thread state`. The function cannot
+be called before Python initialization nor after Python finalization.
 
  .. versionadded:: 3.14
 

Doc/c-api/memory.rst

@@ -110,12 +110,12 @@ The three allocation domains are:
 
 * Raw domain: intended for allocating memory for general-purpose memory
  buffers where the allocation *must* go to the system allocator or where the
- allocator can operate without the :term:`GIL`. The memory is requested directly
- from the system. See :ref:`Raw Memory Interface <raw-memoryinterface>`.
+ allocator can operate without an :term:`attached thread state`. The memory
+is requested directly from the system. See :ref:`Raw Memory Interface <raw-memoryinterface>`.
 
 * "Mem" domain: intended for allocating memory for Python buffers and
  general-purpose memory buffers where the allocation must be performed with
-the :term:`GIL` held. The memory is taken from the Python private heap.
+an :term:`attached thread state`. The memory is taken from the Python private heap.
  See :ref:`Memory Interface <memoryinterface>`.
 
 * Object domain: intended for allocating memory for Python objects. The
@@ -139,8 +139,8 @@ Raw Memory Interface
 ====================
 
 The following function sets are wrappers to the system allocator. These
-functions are thread-safe, the :term:`GIL <global interpreter lock>` does not
-need to be held.
+functions are thread-safe, so a :term:`thread state` does not
+need to be :term:`attached <attached thread state>`.
 
 The :ref:`default raw memory allocator <default-memory-allocators>` uses
 the following functions: :c:func:`malloc`, :c:func:`calloc`, :c:func:`realloc`
@@ -213,8 +213,7 @@ The :ref:`default memory allocator <default-memory-allocators>` uses the
 
 .. warning::
 
- The :term:`GIL <global interpreter lock>` must be held when using these
- functions.
+ There must be an :term:`attached thread state` when using these functions.
 
 .. versionchanged:: 3.6
 
@@ -327,8 +326,7 @@ The :ref:`default object allocator <default-memory-allocators>` uses the
 
 .. warning::
 
- The :term:`GIL <global interpreter lock>` must be held when using these
- functions.
+ There must be an :term:`attached thread state` when using these functions.
 
 .. c:function:: void* PyObject_Malloc(size_t n)
 
@@ -485,12 +483,12 @@ Customize Memory Allocators
  zero bytes.
 
  For the :c:macro:`PYMEM_DOMAIN_RAW` domain, the allocator must be
- thread-safe: the :term:`GIL <global interpreter lock>` is not held when the
- allocator is called.
+ thread-safe: a :term:`thread state` is not :term:`attached <attached thread state>`
+when the allocator is called.
 
  For the remaining domains, the allocator must also be thread-safe:
  the allocator may be called in different interpreters that do not
- share a ``GIL``.
+ share a :term:`GIL`.
 
  If the new allocator is not a hook (does not call the previous allocator),
  the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the
@@ -507,8 +505,8 @@ Customize Memory Allocators
  :c:func:`Py_InitializeFromConfig` to install a custom memory
  allocator. There are no restrictions over the installed allocator
  other than the ones imposed by the domain (for instance, the Raw
- Domain allows the allocator to be called without the GIL held). See
- :ref:`the section on allocator domains <allocator-domains>` for more
+ Domain allows the allocator to be called without an :term:`attached thread state`).
+See :ref:`the section on allocator domains <allocator-domains>` for more
  information.
 
  * If called after Python has finish initializing (after
@@ -555,7 +553,7 @@ Runtime checks:
  called on a memory block allocated by :c:func:`PyMem_Malloc`.
 - Detect write before the start of the buffer (buffer underflow).
 - Detect write after the end of the buffer (buffer overflow).
-- Check that the :term:`GIL <global interpreter lock>` is held when
+- Check that there is an :term:`attached thread state` when
  allocator functions of :c:macro:`PYMEM_DOMAIN_OBJ` (ex:
  :c:func:`PyObject_Malloc`) and :c:macro:`PYMEM_DOMAIN_MEM` (ex:
  :c:func:`PyMem_Malloc`) domains are called.
@@ -620,8 +618,8 @@ PYMEM_CLEANBYTE (meaning uninitialized memory is getting used).
  The :c:func:`PyMem_SetupDebugHooks` function now also works on Python
  compiled in release mode. On error, the debug hooks now use
  :mod:`tracemalloc` to get the traceback where a memory block was allocated.
- The debug hooks now also check if the GIL is held when functions of
- :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are
+ The debug hooks now also check if there is an :term:`attached thread state` when
+functions of :c:macro:`PYMEM_DOMAIN_OBJ` and :c:macro:`PYMEM_DOMAIN_MEM` domains are
  called.
 
 .. versionchanged:: 3.8

Doc/c-api/module.rst

@@ -709,7 +709,7 @@ since multiple such modules can be created from a single definition.
  mechanisms (either by calling it directly, or by referring to its
  implementation for details of the required state updates).
 
- The caller must hold the GIL.
+ The caller must have an :term:`attached thread state`.
 
  Return ``-1`` with an exception set on error, ``0`` on success.
 
@@ -720,6 +720,6 @@ since multiple such modules can be created from a single definition.
  Removes the module object created from *def* from the interpreter state.
  Return ``-1`` with an exception set on error, ``0`` on success.
 
- The caller must hold the GIL.
+ The caller must have an :term:`attached thread state`.
 
  .. versionadded:: 3.3

Doc/c-api/perfmaps.rst

@@ -16,7 +16,7 @@ kernel/git/torvalds/linux.git/tree/tools/perf/Documentation/jit-interface.txt>`_
 In Python, these helper APIs can be used by libraries and features that rely
 on generating machine code on the fly.
 
-Note that holding the Global Interpreter Lock (GIL) is not required for these APIs.
+Note that holding an :term:`attached thread state` is not required for these APIs.
 
 .. c:function:: int PyUnstable_PerfMapState_Init(void)
 

Doc/c-api/reflection.rst

@@ -55,7 +55,7 @@ Reflection
 
 .. c:function:: PyFrameObject* PyEval_GetFrame(void)
 
- Return the current thread state's frame, which is ``NULL`` if no frame is
+ Return the :term:`attached thread state`'s frame, which is ``NULL`` if no frame is
  currently executing.
 
  See also :c:func:`PyThreadState_GetFrame`.

Doc/c-api/sys.rst

@@ -232,7 +232,7 @@ Operating System Utilities
 
  The file descriptor is created non-inheritable (:pep:`446`).
 
- The caller must hold the GIL.
+ The caller must have an :term:`attached thread state`.
 
  .. versionadded:: 3.14
 
@@ -378,8 +378,8 @@ accessible to C code. They all work with the current interpreter thread's
  silently abort the operation by raising an error subclassed from
  :class:`Exception` (other errors will not be silenced).
 
- The hook function is always called with the GIL held by the Python
- interpreter that raised the event.
+ The hook function is always called with an :term:`attached thread state` by
+the Python interpreter that raised the event.
 
  See :pep:`578` for a detailed description of auditing. Functions in the
  runtime and standard library that raise events are listed in the

Doc/c-api/time.rst

@@ -56,7 +56,7 @@ range.
 system time.)
 
 As any other C API (unless otherwise specified), the functions must be called
-with the :term:`GIL` held.
+with an :term:`attached thread state`.
 
 .. c:function:: int PyTime_Monotonic(PyTime_t *result)
 
@@ -78,29 +78,29 @@ Raw Clock Functions
 -------------------
 
 Similar to clock functions, but don't set an exception on error and don't
-require the caller to hold the GIL.
+require the caller to have an :term:`attached thread state`.
 
 On success, the functions return ``0``.
 
 On failure, they set ``*result`` to ``0`` and return ``-1``, *without* setting
-an exception. To get the cause of the error, acquire the GIL and call the
-regular (non-``Raw``) function. Note that the regular function may succeed after
+an exception. To get the cause of the error, :term:`attach <attached thread state>` a :term:`thread state`,
+and call the regular (non-``Raw``) function. Note that the regular function may succeed after
 the ``Raw`` one failed.
 
 .. c:function:: int PyTime_MonotonicRaw(PyTime_t *result)
 
  Similar to :c:func:`PyTime_Monotonic`,
- but don't set an exception on error and don't require holding the GIL.
+ but don't set an exception on error and don't require an :term:`attached thread state`.
 
 .. c:function:: int PyTime_PerfCounterRaw(PyTime_t *result)
 
  Similar to :c:func:`PyTime_PerfCounter`,
- but don't set an exception on error and don't require holding the GIL.
+ but don't set an exception on error and don't require an :term:`attached thread state`.
 
 .. c:function:: int PyTime_TimeRaw(PyTime_t *result)
 
  Similar to :c:func:`PyTime_Time`,
- but don't set an exception on error and don't require holding the GIL.
+ but don't set an exception on error and don't require an :term:`attached thread state`.
 
 
 Conversion functions

Doc/c-api/typeobj.rst

@@ -731,7 +731,7 @@ and :c:data:`PyType_Type` effectively act as defaults.)
  object becomes part of a refcount cycle, that cycle might be collected by
  a garbage collection on any thread). This is not a problem for Python
  API calls, since the thread on which :c:member:`!tp_dealloc` is called
-will own the Global Interpreter Lock (GIL). However, if the object being
+with an :term:`attached thread state`. However, if the object being
  destroyed in turn destroys objects from some other C or C++ library, care
  should be taken to ensure that destroying those objects on the thread
  which called :c:member:`!tp_dealloc` will not violate any assumptions of

Doc/extending/newtypes_tutorial.rst

@@ -403,8 +403,8 @@ the new attribute values. We might be tempted, for example to assign the
 But this would be risky. Our type doesn't restrict the type of the
 ``first`` member, so it could be any kind of object. It could have a
 destructor that causes code to be executed that tries to access the
-``first`` member; or that destructor could release the
-:term:`Global interpreter Lock <GIL>` and let arbitrary code run in other
+``first`` member; or that destructor could detach the
+:term:`thread state <attached thread state>` and let arbitrary code run in other
 threads that accesses and modifies our object.
 
 To be paranoid and protect ourselves against this possibility, we almost
@@ -413,8 +413,8 @@ don't we have to do this?
 
 * when we absolutely know that the reference count is greater than 1;
 
-* when we know that deallocation of the object [#]_ will neither release
- the :term:`GIL` nor cause any calls back into our type's code;
+* when we know that deallocation of the object [#]_ will neither detach
+ the :term:`thread state <attached thread state>` nor cause any calls back into our type's code;
 
 * when decrementing a reference count in a :c:member:`~PyTypeObject.tp_dealloc`
  handler on a type which doesn't support cyclic garbage collection [#]_.

Doc/glossary.rst

@@ -132,6 +132,28 @@ Glossary
  iterator's :meth:`~object.__anext__` method until it raises a
  :exc:`StopAsyncIteration` exception. Introduced by :pep:`492`.
 
+ attached thread state
+
+ A :term:`thread state` that is active for the current OS thread.
+
+ When a :term:`thread state` is attached, the OS thread has
+ access to the full Python C API and can safely invoke the
+ bytecode interpreter.
+
+ Unless a function explicitly notes otherwise, attempting to call
+ the C API without an attached thread state will result in a fatal
+ error or undefined behavior. A thread state can be attached and detached
+ explicitly by the user through the C API, or implicitly by the runtime,
+ including during blocking C calls and by the bytecode interpreter in between
+ calls.
+
+ On most builds of Python, having an attached thread state implies that the
+ caller holds the :term:`GIL` for the current interpreter, so only
+ one OS thread can have an attached thread state at a given moment. In
+ :term:`free-threaded <free threading>` builds of Python, threads can concurrently
+ hold an attached thread state, allowing for true parallelism of the bytecode
+ interpreter.
+
  attribute
  A value associated with an object which is usually referenced by name
  using dotted expressions.
@@ -622,6 +644,10 @@ Glossary
  multi-threaded applications and makes it easier to use multi-core CPUs
  efficiently. For more details, see :pep:`703`.
 
+ In prior versions of Python's C API, a function might declare that it
+ requires the GIL to be held in order to use it. This refers to having an
+ :term:`attached thread state`.
+
  hash-based pyc
  A bytecode cache file that uses the hash rather than the last-modified
  time of the corresponding source file to determine its validity. See
@@ -1295,6 +1321,29 @@ Glossary
  See also :term:`binary file` for a file object able to read and write
  :term:`bytes-like objects <bytes-like object>`.
 
+ thread state
+
+ The information used by the :term:`CPython` runtime to run in an OS thread.
+ For example, this includes the current exception, if any, and the
+ state of the bytecode interpreter.
+
+ Each thread state is bound to a single OS thread, but threads may have
+ many thread states available. At most, one of them may be
+ :term:`attached <attached thread state>` at once.
+
+ An :term:`attached thread state` is required to call most
+ of Python's C API, unless a function explicitly documents otherwise.
+ The bytecode interpreter only runs under an attached thread state.
+
+ Each thread state belongs to a single interpreter, but each interpreter
+ may have many thread states, including multiple for the same OS thread.
+ Thread states from multiple interpreters may be bound to the same
+ thread, but only one can be :term:`attached <attached thread state>` in
+ that thread at any given moment.
+
+ See :ref:`Thread State and the Global Interpreter Lock <threads>` for more
+ information.
+
  token
 
  A small unit of source code, generated by the