gh-119182: Add PyUnicodeWriter C API

[vstinner]

• Issue: [C API] Add an efficient public PyUnicodeWriter API #119182

[vstinner]

@erlend-aasland@serhiy-storchaka@encukou: Do you want to review this change?

[erlend-aasland]

Nit; I prefer to use SemBr for paragraphs like this.

Suggested change

	*str* must be Python :class:`str` object. *start* must be greater than or
	equal to 0, and less than or equal to *end*. *end* must be less than or
	equal to *str* length.
	*str* must be Python :class:`str` object.
	*start* must be greater than or equal to 0,
	and less than or equal to *end*.
	*end* must be less than or equal to *str* length.

[serhiy-storchaka]

TIL that this is called SemBr!

Breaking on comma may be too much, but I prefer to break at the sentence boundary.

[erlend-aasland]

Yeah. You don't need to break at comma, but I often do to minimise future diffs.

Alternative suggestion:

Suggested change

	*str* must be Python :class:`str` object. *start* must be greater than or
	equal to 0, and less than or equal to *end*. *end* must be less than or
	equal to *str* length.
	*str* must be Python :class:`str` object.
	*start* must be greater than or equal to 0, and less than or equal to *end*.
	*end* must be less than or equal to *str* length.

[vstinner]

@erlend-aasland: I addressed your review. Would you mind to review the updated PR?

[serhiy-storchaka]

Except using TypeError and ValueError instead of SystemError, LGTM.

But I do not insist if other core developers prefer TypeError and ValueError. I just think that SystemError is more appropriate for programming errors in using the C API.

[malemburg]

A more general question in the light of the free threading patches: Are these writer APIs thread-safe ?

[vstinner]

A more general question in the light of the free threading patches: Are these writer APIs thread-safe ?

I did nothing to make them safe, so I would say that they are not thread safe. You must not share a writer between two threads. Use one writer per thread.

[vstinner]

@malemburg@serhiy-storchaka: I modified the implementation to make write operations atomic. Either the whole string is written, or "nothing is written".

In the doc, I wrote "On error, leave the writer unchanged". In practice, it's more subtle, the internal buffer can be enlarged, or its kind (UCS1, UCS2 or UCS4) can change. But from the API consumer point of view, it's as if nothing was written.

I added two unit tests to show that it's still possible to use a writer after an error.

The hack is just to save/restore writer->pos internally in the 2 functions which are not atomic: WriteUTF8() and WriteFormat().

I withdraw my approve because writing cannot be atomic.

[erlend-aasland]

Nice; thanks!

[erlend-aasland]

Nit: inconsistent with the rest of the PR.

Suggested change

	PyObject*
	PyObject*

[picnixz]

I also took the liberty of exhaustively point where the 2-blank lines rule is missing (the file is huge so committing directly might help).

[vstinner]

@picnixz: I appreciate your remarks about coding style, but IMO you're going too far. I marked your "two empty lines" suggestions as resolved. This PR is already approved, twice. It's a complex API being discussed for 1 month and I would prefer to not waste my time on coding style. However, I fixed the "an Unicode" typos, thanks :-)

[vstinner]

The C API Working Group agreed to start with this minimum PyUnicodeWriter API. I merged my PR.