gh-128863: Deprecate the private _PyUnicodeWriter API

[vstinner]

Deprecate private C API functions:

• _PyUnicodeWriter_Init()
• _PyUnicodeWriter_Finish()
• _PyUnicodeWriter_Dealloc()
• _PyUnicodeWriter_WriteChar()
• _PyUnicodeWriter_WriteStr()
• _PyUnicodeWriter_WriteSubstring()
• _PyUnicodeWriter_WriteASCIIString()
• _PyUnicodeWriter_WriteLatin1String()

These functions are not deprecated in the internal C API (if the Py_BUILD_CORE macro is defined).

• Issue: [C API] Deprecate private functions which have a public replacement #128863

---

📚 Documentation preview 📚: https://cpython-previews--129245.org.readthedocs.build/

[vstinner]

cc @serhiy-storchaka

[vstinner]

cc @erlend-aasland

[serhiy-storchaka]

LGTM.

I expected a large PR that replaces the _PyUnicodeWriter API with the PyUnicodeWriter API.

[vstinner]

I expected a large PR that replaces the _PyUnicodeWriter API with the PyUnicodeWriter API.

I wrote a serie of changes to use the public C API in issue gh-119182:

• gh-119182: Use public PyUnicodeWriter in wrap_strftime() #129206
• gh-119182: Use public PyUnicodeWriter in time_strftime() #129207
• gh-119182: Use public PyUnicodeWriter in ast_unparse.c #129208
• gh-119182: Use public PyUnicodeWriter in Python-ast.c #129209
• gh-119182: Use public PyUnicodeWriter in stringio.c #129243
• gh-119182: Use public PyUnicodeWriter in _json.c #129249

[vstinner]

@encukou: The replacement functions are straightforward, except for _PyUnicodeWriter_Init() which requires to pass an argument to PyUnicodeWriter_Create(). I documented the replacement as PyUnicodeWriter_Create(0).

[encukou]

I do not see the benefit of removing these.

[bedevere-app]

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

[vstinner]

I do not see the benefit of removing these.

The private _PyUnicodeWriter API has low-level funtions such as _PyUnicodeWriter_Prepare(writer, length, maxchar) and _PyUnicodeWriter_PrepareKind(writer, kind) which are specific to PEP 393 internals. I would like to get rid of these to get more freedom on how the public PyUnicodeWriter API is implemented (since it's currently implemented on top of the private API).

The private API has no documentation and is not tested. Removing it reduces the size of the C API.

It's also a way to promote the public C API which doesn't expose PEP 393 internals.

[encukou]

Do you think it'd be a good idea to mention the reasons in the porting notes? If people see the reason, maybe the issues they file will be less angry.

I would like to get rid of these to get more freedom on how the public PyUnicodeWriter API is implemented (since it's currently implemented on top of the private API).

If that's the reason, is planning to remove them in 3.18 a good idea?
If old private API actually blocks current development, we should be able to remove it immediately (in the version where it's blocking something).

[vstinner]

Do you think it'd be a good idea to mention the reasons in the porting notes? If people see the reason, maybe the issues they file will be less angry.

We don't document rationale for deprecations in the documentation. I don't think that it's worth it.

If that's the reason, is planning to remove them in 3.18 a good idea? If old private API actually blocks current development, we should be able to remove it immediately (in the version where it's blocking something).

Changing PyUnicodeWriter implementation would be nice, but I don't think that it should urge to remove the private API right now, it can wait.

[encukou]

Changing PyUnicodeWriter implementation would be nice, but I don't think that it should urge to remove the private API right now, it can wait.

Why can't the deprecation/removal also wait?

[vstinner]

Why can't the deprecation/removal also wait?

If the function is never deprecated, it will only postpone when the private C API can be modified/enhanced.

[erlend-aasland]

I do not see the benefit of removing these.

It's a deprecation, not a removal.

[encukou]

It's a deprecation, not a removal.

This PR schedules a removal for 3.18.
I don't think that helps the goal here (as far as I understand the goal). I commented at https://discuss.python.org/t/68081/15 .

[vstinner]

I created a C API Working Group decision issue for this change: capi-workgroup/decisions#57

[vstinner]

I created a C API Working Group decision issue for this change: capi-workgroup/decisions#57

The working group decided to deprecate the API and remove it in Python 3.18.

Petr wrote "Count me as -0 so I don't block the work" on the working group issue.