gh-131591: Implement PEP 768

Doc/library/sys.rst

@@ -1835,6 +1835,28 @@ always available. Unless explicitly noted otherwise, all variables are read-only
 
  .. versionadded:: 3.12
 
+
+.. function:: remote_exec(pid, script)
+
+ Executes a file containing Python code given by *script* in the remote
+ process with the given *pid*.
+
+ This function returns immediately, and the code will be executed by the
+ target process's main thread at the next available opportunity, similarly
+ to how signals are handled. There is no interface to determine when the
+ code has been executed. The caller is responsible for making sure that
+ the file still exists whenever the remote process tries to read it and that
+ it hasn't been overwritten.
+
+ The remote process must be running a CPython interpreter of the same major
+ and minor version as the local process. If either the local or remote
+ interpreter is pre-release (alpha, beta, or release candidate) then the
+ local and remote interpreters must be the same exact version.
+
+ .. availability:: Unix, Windows.
+ .. versionadded:: 3.14
+
+
 .. function:: _enablelegacywindowsfsencoding()
 
  Changes the :term:`filesystem encoding and error handler` to 'mbcs' and

Doc/using/cmdline.rst

@@ -603,6 +603,13 @@ Miscellaneous options
 
  .. versionadded:: 3.13
 
+ * ``-X disable_remote_debug`` disables the remote debugging support as described
+ in :pep:`768`. This option is only available on some platforms and will do nothing
+ if is not supported on the current system. See also
+ :envvar:`PYTHON_DISABLE_REMOTE_DEBUG` and :pep:`768`.
+
+ .. versionadded:: 3.14
+
  * :samp:`-X cpu_count={n}` overrides :func:`os.cpu_count`,
  :func:`os.process_cpu_count`, and :func:`multiprocessing.cpu_count`.
  *n* must be greater than or equal to 1.
@@ -1160,7 +1167,14 @@ conflict.
 
  .. versionadded:: 3.13
 
+.. envvar:: PYTHON_DISABLE_REMOTE_DEBUG
+
+ If this variable is set to a non-empty string, it disables the remote
+ debugging feature described in :pep:`768`.
+
+ See also the :option:`-X disable_remote_debug` command-line option.
 
+ .. versionadded:: 3.14
 
 .. envvar:: PYTHON_CPU_COUNT
 

Doc/using/configure.rst

@@ -660,6 +660,15 @@ also be used to improve performance.
  Add ``-fstrict-overflow`` to the C compiler flags (by default we add
  ``-fno-strict-overflow`` instead).
 
+.. option:: --without-remote-debug
+
+ Deactivate remote debugging support described in :pep:`768` (enabled by default).
+ When this flag is provided the code that allows the interpreter to schedule the
+ execution of a Python file in a separate process as described in :pep:`768` is
+ not compiled.
+
+ ..versionadded:: 3.14
+
 
 .. _debug-build:
 

Doc/whatsnew/3.14.rst

@@ -90,6 +90,79 @@
 New features
 ============
 
+.. _whatsnew314-pep678:
+
+PEP 768: Safe external debugger interface for CPython
+----------------------------------------------------
+
+:pep:`768` introduces a zero-overhead debugging interface that allows debuggers and profilers
+to safely attach to running Python processes. This is a significant enhancement to Python's
+debugging capabilities, bringing them in line with other major programming languages.
+
+The new interface provides safe execution points for attaching debugger code without modifying
+the interpreter's normal execution path or adding runtime overhead. This enables tools to
+inspect and interact with Python applications in real-time without stopping or restarting
+them — a crucial capability for high-availability systems and production environments.
+
+For convenience, CPython implements this interface through the :mod:`sys` module with a
+:func:`sys.remote_exec` function::
+
+ sys.remote_exec(pid, script_path)
+
+This function allows sending Python code to be executed in a target process at the next safe
+execution point. However, tool authors can also implement the protocol directly as described
+in the PEP, which details the underlying mechanisms used to safely attach to running processes.
+
+Here's a simple example that inspects object types in a running Python process:
+
+ .. code-block:: python
+
+ import sys
+ import tempfile
+ import os
+
+ # Create a temporary script
+ with tempfile.NamedTemporaryFile(mode='w', suffix='.py', delete=False) as f:
+ script_path = f.name
+ f.write("""
+ import gc
+ import collections
+
+ # Collect all objects managed by the garbage collector
+ gc.collect()
+
+ # Count objects by type
+ type_counts = collections.Counter(type(obj).__name__
+ for obj in gc.get_objects())
+
+ # Print the most common types
+ print("Most common object types in process:")
+ for type_name, count in type_counts.most_common(10):
+ print(f"{type_name}:{count}")
+ """)
+
+ try:
+ # Execute in process with PID 1234
+ print("Behold! An offering:")
+ sys.remote_exec(1234, script_path)
+ finally:
+ os.unlink(script_path)
+
+The debugging interface has been carefully designed with security in mind and includes several
+mechanisms to control access:
+
+* A :envvar:`PYTHON_DISABLE_REMOTE_DEBUG` environment variable.
+* A :option:`-X disable-remote-debug` command-line option.
+* A ``--without-remote-debug`` configure flag to completely disable the feature at build time.
+
+A key implementation detail is that the interface piggybacks on the interpreter's existing evaluation
+loop and safe points, ensuring zero overhead during normal execution while providing a reliable way
+for external processes to coordinate debugging operations.
+
+See :pep:`768` for more details.
+
+(Contributed by Pablo Galindo Salgado, Matt Wozniski, and Ivona Stojanovic in :gh:`131591`.)
+
 .. _whatsnew314-pep649:
 
 PEP 649: deferred evaluation of annotations

Include/cpython/initconfig.h

@@ -143,6 +143,7 @@ typedef struct PyConfig{
  int faulthandler;
  int tracemalloc;
  int perf_profiling;
+ int remote_debug;
  int import_time;
  int code_debug_ranges;
  int show_ref_count;

Include/cpython/pystate.h

@@ -29,6 +29,13 @@ typedef int (*Py_tracefunc)(PyObject *, PyFrameObject *, int, PyObject *);
 #define PyTrace_C_RETURN 6
 #define PyTrace_OPCODE 7
 
+/* Remote debugger support */
+# define MAX_SCRIPT_PATH_SIZE 512
+typedef struct _remote_debugger_support{
+ int32_t debugger_pending_call;
+ char debugger_script_path[MAX_SCRIPT_PATH_SIZE];
+} _PyRemoteDebuggerSupport;
+
 typedef struct _err_stackitem{
  /* This struct represents a single execution context where we might
  * be currently handling an exception. It is a per-coroutine state
@@ -202,6 +209,7 @@ struct _ts{
  The PyThreadObject must hold the only reference to this value.
  */
  PyObject *threading_local_sentinel;
+ _PyRemoteDebuggerSupport remote_debugger_support;
 };
 
 # define Py_C_RECURSION_LIMIT 5000

Include/internal/pycore_debug_offsets.h

@@ -73,6 +73,7 @@ typedef struct _Py_DebugOffsets{
  uint64_t id;
  uint64_t next;
  uint64_t threads_head;
+ uint64_t threads_main;
  uint64_t gc;
  uint64_t imports_modules;
  uint64_t sysdict;
@@ -206,6 +207,15 @@ typedef struct _Py_DebugOffsets{
  uint64_t gi_iframe;
  uint64_t gi_frame_state;
  } gen_object;
+
+ struct _debugger_support{
+ uint64_t eval_breaker;
+ uint64_t remote_debugger_support;
+ uint64_t remote_debugging_enabled;
+ uint64_t debugger_pending_call;
+ uint64_t debugger_script_path;
+ uint64_t debugger_script_path_size;
+ } debugger_support;
 } _Py_DebugOffsets;
 
 
@@ -223,6 +233,7 @@ typedef struct _Py_DebugOffsets{
  .id = offsetof(PyInterpreterState, id), \
  .next = offsetof(PyInterpreterState, next), \
  .threads_head = offsetof(PyInterpreterState, threads.head), \
+ .threads_main = offsetof(PyInterpreterState, threads.main), \
  .gc = offsetof(PyInterpreterState, gc), \
  .imports_modules = offsetof(PyInterpreterState, imports.modules), \
  .sysdict = offsetof(PyInterpreterState, sysdict), \
@@ -326,6 +337,14 @@ typedef struct _Py_DebugOffsets{
  .gi_iframe = offsetof(PyGenObject, gi_iframe), \
  .gi_frame_state = offsetof(PyGenObject, gi_frame_state), \
  }, \
+ .debugger_support ={\
+ .eval_breaker = offsetof(PyThreadState, eval_breaker), \
+ .remote_debugger_support = offsetof(PyThreadState, remote_debugger_support), \
+ .remote_debugging_enabled = offsetof(PyInterpreterState, config.remote_debug), \
+ .debugger_pending_call = offsetof(_PyRemoteDebuggerSupport, debugger_pending_call), \
+ .debugger_script_path = offsetof(_PyRemoteDebuggerSupport, debugger_script_path), \
+ .debugger_script_path_size = MAX_SCRIPT_PATH_SIZE, \
+ }, \
 }
 
 

Include/internal/pycore_global_strings.h

@@ -686,6 +686,7 @@ struct _Py_global_strings{
  STRUCT_FOR_ID(salt)
  STRUCT_FOR_ID(sched_priority)
  STRUCT_FOR_ID(scheduler)
+ STRUCT_FOR_ID(script)
  STRUCT_FOR_ID(second)
  STRUCT_FOR_ID(security_attributes)
  STRUCT_FOR_ID(seek)

Include/internal/pycore_sysmodule.h

@@ -24,6 +24,8 @@ extern int _PySys_ClearAttrString(PyInterpreterState *interp,
 extern int _PySys_SetFlagObj(Py_ssize_t pos, PyObject *new_value);
 extern int _PySys_SetIntMaxStrDigits(int maxdigits);
 
+extern int _PySysRemoteDebug_SendExec(int pid, int tid, const char *debugger_script_path);
+
 #ifdef __cplusplus
 }
 #endif