Can't import `concurrent.futures.thread` at shutdown

[pitrou]

Bug report

Bug description:

Simple reproducer (real-world bug report is a bit more complex):

import threading


def use_cf():
 import concurrent.futures.thread


def main():
 t = threading.Thread(target=use_cf)
 t.start()


if __name__ == "__main__":
 main()

CPython versions tested on:

3.10, 3.11, 3.12, 3.13

Operating systems tested on:

No response

[pitrou]

I don't know if we want to do anything about this, I'm just posting just in case.

@gpshead

[gpshead]

In general we consider this working as intended (from the linked bug):

$ python main.py 
Traceback (most recent call last):
 File "pyarrow/pandas-shim.pxi", line 100, in pyarrow.lib._PandasAPIShim._check_import
 File "pyarrow/pandas-shim.pxi", line 48, in pyarrow.lib._PandasAPIShim._import_pandas
 File "/home/jleibs/pyarrow-repro/venv/lib/python3.10/site-packages/pyarrow/pandas_compat.py", line 24, in <module>
 import concurrent.futures.thread # noqa
 File "/usr/lib/python3.10/concurrent/futures/thread.py", line 37, in <module>
 threading._register_atexit(_python_exit)
 File "/usr/lib/python3.10/threading.py", line 1504, in _register_atexit
 raise RuntimeError("can't register atexit after shutdown")
RuntimeError: can't register atexit after shutdown

After the main interpreter thread has exited, launching new threads isn't really supported.

This has been true since 3.9 thanks to b61b818 which resolved Issue #83993 via PR #19149.

[pitrou]

The problem isn't launching new threads here, it's simply importing the module. In other words, c.f.thread is imported as a side effect of the user importing another module after shutdown has started, even though the executor functionality wouldn't have been used anyway. It is of course possible to workaround by catching the RuntimeError, but it's a bit of a pointless annoyance.

[pitrou]

In general, I think it's a good idea for module import to be graceful, even though it can be complicated to achieve in some cases (and concurrent.futures is of course a complex module).

[gpshead]

It isn't reasonable to expect all non-trivial imports to work in a finalization context.

…

On Thu, Dec 14, 2023, 1:19 AM Antoine Pitrou ***@***.***> wrote: In general, I think it's a good idea for module import to be graceful, even though it can be complicated to achieve in some cases (and concurrent.futures is of course a complex module). — Reply to this email directly, view it on GitHub <#113060 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAQXCZ7H5M23XR63EPF57LYJLAARAVCNFSM6AAAAABATUVWJ6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNJVGQ3TAMZVGE> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>