gh-117398: datetime: Access C-API via PyInterpreterState

[neonene]

An alternative PR to #117413, which also makes the datetime C-API available to the isolated subinterpreters.

The pointer to the PyDateTime_CAPI struct is stored in PyInterpreterState right after the PyCapsule_Import() call. This version does not change Modules/_testcapi/datetime.cfor now. Also, the overhead should be even across interpreters. However, this extends the PyDateTime_CAPI struct.

cc @pganssle@vstinner@ericsnowcurrently@encukou@erlend-aasland

• Issue: Isolate the _datetime extension module #117398

[vstinner]

I would prefer to not leak such implementation detail at the ABI level, and so use a regular function rather than a static inline function. I would prefer that _pydatetime_capi is private.

[neonene]

How can I resolve non-Windows errors? I'm not sure how to use private struct members here.

Error: _testcapi failed to import: /home/runner/work/cpython/cpython-builddir/ build/lib.linux-x86_64-3.13-pydebug/_testcapi.cpython-313d-x86_64-linux-gnu.so: undefined symbol: _PyDateTimeAPI_Get 

[neonene]

I added another capsule which points to the _PyDateTimeAPI_Import function in _datetimemodule.c. Any buildable alternative is welcome.

[neonene]

I have mixed this patch into the multi-phase init PR: #118337.

@encukou Is there any chance that users could test this or #117413 in the beta phase? If these approaches work, I think _detetime can be isolated in a regular way.

[vstinner]

Such change is too late for beta1. It's better to wait for the beginning of the 3.14 dev cycle.

[neonene]

Such change is too late for beta1.

How do you think of the situation?

[vstinner]

In my experience, such change is likely to cause regression and need more time to be stabilized. I prefer to wait Wednesday.

[neonene]

Otherwise, 3.13 should take specific workarounds for the crash during after the interned-strings cleanup.

[neonene]

A workaround is ready for review: #118531

[vstinner]

Otherwise, 3.13 should take specific workarounds for the crash during after the interned-strings cleanup.

Which crash? This PR is related to issue gh-117398 which doesn't mention any crash.

[ericsnowcurrently]

FYI, I asked @Yhg1s if we could get an exception for landing this after beta 1: #103092 (comment). He gave us a similar exception for 3.12. Thomas might not do that this time, but at least I've asked.

[Yhg1s]

FYI, I asked @Yhg1s if we could get an exception for landing this after beta 1: #103092 (comment). He gave us a similar exception for 3.12. Thomas might not do that this time, but at least I've asked.

This particular change is just one that adds a new capsule C API, which can then be used when in the future the datetime module switches to multi-phase init and better subinterpreter support, right? Given that it's of mild impact and it gives us an API that allows us to deprecate the old one and use multi-phase init for datetime, I'm okay with this going into beta 2.

[vstinner]

It doesn't sound safe to me to announce that the _datetime extension is compatible with subinterpreters before static types are converted to heap types. I suggest to discuss the plan in the issue: #117398 (comment)

[neonene]

@vstinner I think this should be tested from 3.13.0b1+. The change is easy to diagnose or revert. If an inevitable problem is found, the case will support your exceptional approach (mix _datetime types into PyInterpreterState) whose rationale is currently not proposed.

EDIT: Ultimately, PyInterpreterState does not need to be used, as the alternative PR proves.

[vstinner]

IMO Py_MOD_PER_INTERPRETER_GIL_SUPPORTED cannot be added before other changes are done, such as converting static types to heap types.

I prefer to do the conversion in a different order, with multiple steps: #117398 (comment)

[bedevere-app]

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

[neonene]

IMO Py_MOD_PER_INTERPRETER_GIL_SUPPORTED cannot be added before other changes are done, such as converting static types to heap types.

This PR does not add the flag in _datetimemodule.c. The test case in _testmultiphase.c is a kind of _zoneinfo multi-phase init module with the flag.