gh-84559: Change the multiprocessing start method default to forkserver

Doc/library/concurrent.futures.rst

@@ -286,14 +286,6 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.
 
  Added the *initializer* and *initargs* arguments.
 
- .. note::
- The default :mod:`multiprocessing` start method
- (see :ref:`multiprocessing-start-methods`) will change away from
- *fork* in Python 3.14. Code that requires *fork* be used for their
- :class:`ProcessPoolExecutor` should explicitly specify that by
- passing a ``mp_context=multiprocessing.get_context("fork")``
- parameter.
-
  .. versionchanged:: 3.11
  The *max_tasks_per_child* argument was added to allow users to
  control the lifetime of workers in the pool.
@@ -310,6 +302,12 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.
  *max_workers* uses :func:`os.process_cpu_count` by default, instead of
  :func:`os.cpu_count`.
 
+ .. versionchanged:: 3.14
+ The default process start method (see
+ :ref:`multiprocessing-start-methods`) changed away from *fork*. If you
+ require the *fork* start method for :class:`ProcessPoolExecutor` you must
+ explicitly pass ``mp_context=multiprocessing.get_context("fork")``.
+
 .. _processpoolexecutor-example:
 
 ProcessPoolExecutor Example

Doc/library/multiprocessing.rst

@@ -124,11 +124,11 @@ to start a process. These *start methods* are
  inherited by the child process. Note that safely forking a
  multithreaded process is problematic.
 
- Available on POSIX systems. Currently the default on POSIX except macOS.
+ Available on POSIX systems.
 
- .. note::
-The default start method will change away from *fork* in Python 3.14.
- Code that requires *fork* should explicitly specify that via
+ .. versionchanged:: 3.14
+This is no longer the default start method on any platform.
+ Code that requires *fork* must explicitly specify that via
  :func:`get_context` or :func:`set_start_method`.
 
  .. versionchanged:: 3.12
@@ -146,9 +146,11 @@ to start a process. These *start methods* are
  side-effect so it is generally safe for it to use :func:`os.fork`.
  No unnecessary resources are inherited.
 
- Available on POSIX platforms which support passing file descriptors
-over Unix pipes such as Linux.
+ Available on POSIX platforms which support passing file descriptors over
+ Unix pipes such as Linux. The default on those.
 
+ .. versionchanged:: 3.14
+ This became the default start method on POSIX platforms.
 
 .. versionchanged:: 3.4
  *spawn* added on all POSIX platforms, and *forkserver* added for
@@ -162,6 +164,13 @@ to start a process. These *start methods* are
  method should be considered unsafe as it can lead to crashes of the
  subprocess as macOS system libraries may start threads. See :issue:`33725`.
 
+.. versionchanged:: 3.14
+
+ On POSIX platforms the default start method was changed from *fork* to
+ *forkserver* to retain the performance but avoid common multithreaded
+ process incompatibilities. See :gh:`84559`.
+
+
 On POSIX using the *spawn* or *forkserver* start methods will also
 start a *resource tracker* process which tracks the unlinked named
 system resources (such as named semaphores or

Doc/whatsnew/3.14.rst

@@ -367,6 +367,14 @@ Deprecated
  as a single positional argument.
  (Contributed by Serhiy Storchaka in :gh:`109218`.)
 
+* :mod:`multiprocessing` and :mod:`concurrent.futures`:
+ The default start method (see :ref:`multiprocessing-start-methods`) changed
+ away from *fork* to *forkserver* on platforms where it was not already
+ *spawn* (Windows & macOS). If you require the threading incompatible *fork*
+ start method you must explicitly specify it when using :mod:`multiprocessing`
+ or :mod:`concurrent.futures` APIs.
+ (Contributed by Gregory P. Smith in :gh:`84559`.)
+
 * :mod:`os`:
  :term:`Soft deprecate <soft deprecated>` :func:`os.popen` and
  :func:`os.spawn* <os.spawnl>` functions. They should no longer be used to

Lib/multiprocessing/context.py

@@ -259,13 +259,12 @@ def get_start_method(self, allow_none=False):
 
  def get_all_start_methods(self):
  """Returns a list of the supported start methods, default first."""
- if sys.platform == 'win32':
- return ['spawn']
- else:
- methods = ['spawn', 'fork'] if sys.platform == 'darwin' else ['fork', 'spawn']
- if reduction.HAVE_SEND_HANDLE:
- methods.append('forkserver')
- return methods
+ default = self._default_context.get_start_method()
+ start_method_names = [default]
+ start_method_names.extend(
+ name for name in _concrete_contexts if name != default
+ )
+ return start_method_names
 
 
 #
@@ -320,14 +319,15 @@ def _check_available(self):
  'spawn': SpawnContext(),
  'forkserver': ForkServerContext(),
  }
- if sys.platform == 'darwin':
- # bpo-33725: running arbitrary code after fork() is no longer reliable
- # on macOS since macOS 10.14 (Mojave). Use spawn by default instead.
- _default_context = DefaultContext(_concrete_contexts['spawn'])
+ # bpo-33725: running arbitrary code after fork() is no longer reliable
+ # on macOS since macOS 10.14 (Mojave). Use spawn by default instead.
+ # gh-84559: We changed everyones default to a thread safeish one in 3.14.
+ if reduction.HAVE_SEND_HANDLE and sys.platform != 'darwin':
+ _default_context = DefaultContext(_concrete_contexts['forkserver'])
  else:
- _default_context = DefaultContext(_concrete_contexts['fork'])
+ _default_context = DefaultContext(_concrete_contexts['spawn'])
 
-else:
+else: # Windows
 
  class SpawnProcess(process.BaseProcess):
  _start_method = 'spawn'

Lib/test/_test_multiprocessing.py

@@ -5553,15 +5553,29 @@ def test_set_get(self):
  multiprocessing.set_start_method(old_method, force=True)
  self.assertGreaterEqual(count, 1)
 
- def test_get_all(self):
+ def test_get_all_start_methods(self):
  methods = multiprocessing.get_all_start_methods()
+ self.assertIn('spawn', methods)
  if sys.platform == 'win32':
  self.assertEqual(methods, ['spawn'])
+ elif sys.platform == 'darwin':
+ self.assertEqual(methods[0], 'spawn') # The default is first.
+ # Whether these work or not, they remain available on macOS.
+ self.assertIn('fork', methods)
+ self.assertIn('forkserver', methods)
  else:
- self.assertTrue(methods == ['fork', 'spawn'] or
- methods == ['spawn', 'fork'] or
- methods == ['fork', 'spawn', 'forkserver'] or
- methods == ['spawn', 'fork', 'forkserver'])
+ # POSIX
+ self.assertIn('fork', methods)
+ if other_methods := set(methods) -{'fork', 'spawn'}:
+ # If there are more than those two, forkserver must be one.
+ self.assertEqual({'forkserver'}, other_methods)
+ # The default is the first method in the list.
+ self.assertIn(methods[0],{'forkserver', 'spawn'},
+ msg='3.14+ default must not be fork')
+ if methods[0] == 'spawn':
+ # Confirm that the current default selection logic prefers
+ # forkserver vs spawn when available.
+ self.assertNotIn('forkserver', methods)
 
  def test_preload_resources(self):
  if multiprocessing.get_start_method() != 'forkserver':

Lib/test/support/__init__.py

@@ -2209,7 +2209,15 @@ def skip_if_broken_multiprocessing_synchronize():
  # bpo-38377: On Linux, creating a semaphore fails with OSError
  # if the current user does not have the permission to create
  # a file in /dev/shm/ directory.
- synchronize.Lock(ctx=None)
+ import multiprocessing
+ synchronize.Lock(ctx=multiprocessing.get_context('fork'))
+ # The explicit fork mp context is required in order for
+ # TestResourceTracker.test_resource_tracker_reused to work.
+ # synchronize creates a new multiprocessing.resource_tracker
+ # process at module import time via the above call in that
+ # scenario. Awkward. This enables gh-84559. No code involved
+ # should have threads at that point so fork() should be safe.
+
  except OSError as exc:
  raise unittest.SkipTest(f"broken multiprocessing SemLock:{exc!r}")
 

Misc/NEWS.d/next/Library/2024-09-19-00-09-48.gh-issue-84559.IrxvQe.rst

@@ -0,0 +1,5 @@
+The default :mod:`multiprocessing` start method on Linux and other POSIX
+systems has been changed away from often unsafe ``"fork"`` to ``"forkserver"``
+(when the platform supports sending file handles over pipes as most do) or
+``"spawn"``. Mac and Windows are unchanged as they already default to
+``"spawn"``.