gh-128863: Deprecate private C API functions

[vstinner]

• _PyBytes_Join()
• _PyDict_GetItemStringWithError()
• _PyDict_Pop()
• _PyThreadState_UncheckedGet()
• _PyUnicode_AsString()
• _Py_HashPointer()
• _Py_fopen_obj()

Replace _Py_HashPointer() with Py_HashPointer().

Remove references to deprecated functions.

• Issue: [C API] Deprecate private functions which have a public replacement #128863

---

📚 Documentation preview 📚: https://cpython-previews--128864.org.readthedocs.build/

[vstinner]

cc @encukou@zooba

[encukou]

This is extra work for both you and for users. I do not see how this would reduce our maintenance burden. I also don't see any other benefit of this PR.

[methane]

Removing them from Python 3.16 seems a little hurried. Unless there is a specific reason to remove it sooner, how about schedule it for removal in Python 3.18?

[vstinner]

Code search on PyPI top 7,500 projects (2024-10-16).

_PyBytes_Join(): 4 projects

• cython (3.0.11)
• dbnd (1.0.25.2)
• mod_wsgi (5.0.1)
• mypy (1.12.0)

_PyDict_GetItemStringWithError(): 3 projects

• catboost (1.2.7)
• mypy (1.12.0)
• numpy (2.1.2)

_PyDict_Pop(): 1 project

• cython (3.0.11)

_PyThreadState_UncheckedGet(): 25 projects

• CoolProp (6.6.0)
• aplr (10.6.3)
• cffi (1.17.1)
• correctionlib (2.6.4)
• cython (3.0.11)
• ddtrace (2.14.2)
• debugpy (1.8.7)
• dlib (19.24.6)
• frozendict (2.4.6)
• mmcif (0.89.0)
• onnx (1.17.0)
• onnx-simplifier (0.4.36)
• onnxoptimizer (0.3.13)
• onnxsim (0.4.36)
• opencc (1.1.9)
• praat_parselmouth (0.4.5)
• ptvsd (4.3.2)
• pybind11 (2.13.6)
• pybind11_global (2.13.6)
• pydevd (3.1.0)
• pydevd_pycharm (243.19420.35)
• pymatching (2.2.1)
• pythonnet (3.0.4)
• vaex-core (4.17.1)
• xatlas (0.0.9)

_PyUnicode_AsString(): 1 project

• cvxopt (1.3.2)

_Py_HashPointer(): 3 projects

• cffi (1.17.1)
• igraph (0.11.6)
• numba (0.60.0)

_Py_fopen_obj(): 1 project

• pybind11 (2.13.6)

I ignored projects including copies of these dependencies such as pybind11 copies.

---

Total: 33 projects

Affected projects (33):

• CoolProp (6.6.0)
• aplr (10.6.3)
• catboost (1.2.7)
• cffi (1.17.1)
• correctionlib (2.6.4)
• cvxopt (1.3.2)
• cython (3.0.11)
• dbnd (1.0.25.2)
• ddtrace (2.14.2)
• debugpy (1.8.7)
• dlib (19.24.6)
• frozendict (2.4.6)
• igraph (0.11.6)
• mmcif (0.89.0)
• mod_wsgi (5.0.1)
• mypy (1.12.0)
• numba (0.60.0)
• numpy (2.1.2)
• onnx (1.17.0)
• onnx-simplifier (0.4.36)
• onnxoptimizer (0.3.13)
• onnxsim (0.4.36)
• opencc (1.1.9)
• praat_parselmouth (0.4.5)
• ptvsd (4.3.2)
• pybind11 (2.13.6)
• pybind11_global (2.13.6)
• pydevd (3.1.0)
• pydevd_pycharm (243.19420.35)
• pymatching (2.2.1)
• pythonnet (3.0.4)
• vaex-core (4.17.1)
• xatlas (0.0.9)

[vstinner]

@methane:

Removing them from Python 3.16 seems a little hurried. Unless there is a specific reason to remove it sooner, how about schedule it for removal in Python 3.18?

I'm fine with using a longer deprecation period for _PyThreadState_UncheckedGet() which is more frequently used.

Other functions are used by less than 5 projects and so can use a faster deprecation period, no?

[methane]

• PyBytes_Join is added in Python 3.14.
• Py_fopen is added in 3.14.

Python 3.13 will reach EOL in 2029. I want to keep it until 2031 (2029+2).

• PyDict_GetItemStringRef is added in 3.13.
• PyDict_Pop is added in 3.13.
• PyThreadState_GetUnchecked is added in 3.13.
• Py_HashPointer is added in 3.13.

Python 3.12 will reach EOL in 2028. I want to keep it until 2030 (2028+2).

Removing _PyUnicode_AsString with minimum deprecation period is OK because PyUnicode_AsUTF8 is added in 3.3.

[vstinner]

Removing them from Python 3.16 seems a little hurried. Unless there is a specific reason to remove it sooner, how about schedule it for removal in Python 3.18?

I updated my PR to postpone the removal to Python 3.18.

[vstinner]

cc @serhiy-storchaka@erlend-aasland

[serhiy-storchaka]

I am not sure that it is worth to go such far in preserving compatibility for private C API. Usually we just changed it at will, it was a fair play.

[vstinner]

I am not sure that it is worth to go such far in preserving compatibility for private C API. Usually we just changed it at will, it was a fair play.

Do you suggest to remove these functions earlier?

[serhiy-storchaka]

Were not some of them already removed?

This is only my grumbling. I would remove them without warning if there is a replacement. You have to ask other core developers if they have reasons to keep the old private C API.

Removing a private C API is not the worst thing that can be happen if you use private C API. It can change semantic -- start returning a strong reference instead of a weak reference or vice versa, start stealing a reference, change the order of parameters, etc.

But if keep them -- your PR LGTM.

[vstinner]

Merged, thanks for review.