bpo-47115: Document which parts of structs are in limited API/stable ABI

[encukou]

Some screenshots of the result:

Opaque struct:

Fully visible structs:


Partial struct (see the issue for details):

Included member of the above:

Non-struct items don't change:

https://bugs.python.org/issue47115

[arhadthedev]

A side question since stable_abi.dat is rewritten anyway: can it be renamed into stable_abi.csv (in a separate PR, for sure) so (1) GitHub will format it as a pretty table, (2) spreadsheet editors like OpenOffice Calc and Microsoft Excel could be used? Or there are third party tools that use it so the renaming will break them?

[vstinner]

By the way, I recently added "The structure is not part of the C API." to:
https://docs.python.org/dev/c-api/frame.html

[encukou]

That's wrong. If it's not part of the API at all, you can't use functions like PyFrame_GetBack that take it as an argument.

[vstinner]

It seems like there is a misunderstanding on what "a structure" means here. Python C API is misleading because it uses short PyFrameObject type name rather than struct PyFrameObject.

In practice in C, "PyFrameObject" is a type name, not a structure. I recently added Include/pytypedefs.h to clarify which type names are always available in the C API (and fix a compiler error), even if their related structures are opaque. I understand that the type (type "name") is part of the C API, but the structure is not.

I'm not sure how https://docs.python.org/dev/c-api/frame.html should be rephrased. The follow this PR, maybe "The structure is opaque" which may become redundant once this PR is merged.

[encukou]

Ah, I see.
This should be explained/avoided: if you don't know the term "opaque structure" you are likely to search for it, but if you don't realize the distinction between "type" and "structure", you'll probably just be confused.

I'll work on the rephrasing next week. Please let me know if you know of other places with similar wording.

[encukou]

I started going through all the structs to make sure the docs make sense.
And I quite like the note for PyInterpreterState: “There are no public members in this structure.” Short and clear! I'll use that for PyFrameObject too.

[vstinner]

Is there a definition of an "opaque structure" in the Python doc? Is there a link from "opaque struct" in then generated doc?

[encukou]

Is there a definition of an "opaque structure" in the Python doc? Is there a link from "opaque struct" in then generated doc?

IMO it's a generally known term, it even has a Wikipedia entry. I don't think the docs need to define it.

[encukou]

@arhadthedev, I'll open an issue for it.

[encukou]

Issue for the renaming (and more): https://bugs.python.org/issue47168

[erlend-aasland]

Nice; I think this doc enhancement will be helpful for both core devs and others.

From what I could see, the Py changes look sensible, but I'm not at all familiar with these doc tools. I'll have another look at this tomorrow.

[encukou]

So, how about this?

(The “structure of the objects used to describe frame objects” is weird, but I'm leaving that out of scope of this PR.)

[vstinner]

“There are no public members in this structure.”

That's explicit. It prevents the confusion about "the structure is not part of the C API" and confusion between "structure" and "type". In practice, an opaque structure ... is an empty structure, right ;-) The only subtle difference is that sizeof() doesn't work on an opaque structure, but I don't think that it's worth it to do into the details, unless we add an entry to the glossary for "opaque structure".

[vstinner]

LGTM.

[erlend-aasland]

Looks good to me also.

It looks to me that this comment is now outdated:

cpython/Doc/tools/extensions/c_annotations.py

Lines 101 to 103 in 4b188ae

	# Stable ABI annotation. These have two forms:
	# Part of the [Stable ABI](link).
	# Part of the [Stable ABI](link) since version X.Y.

Either update it or delete it :)

[encukou]

Comment adjusted, and one more tweak to the wording:

(It should be possible to make these -- PyObject & PyVarObject -- fully opaque in the 3.12 limited API.)