gh-91744: Add semi-stable C API tier#91789

encukou · 2022-04-21T15:56:45Z

This adds the semi-stable API tier witth a few initial functions/types. Adding more is left to another PR.

d'oh!

vstinner

Is PyInterpreterState_SetEvalFrameFunc() really usable in the semi-stable API? The second parameter type comes from the internal C API. There is nothing to inspect a _PyInterpreterFrame structure in the semi-stable API.

I don't know how debuggers use PyInterpreterState_SetEvalFrameFunc().

cc @markshannon

vstinner · 2022-04-21T16:54:40Z

See also #91906

h-vetinari

Some comments from taking a look out of curiosity

Include/README.rst

Doc/whatsnew/3.11.rst

h-vetinari · 2022-04-28T03:23:06Z

Doc/c-api/stable.rst

+There are two tiers of API with different stability exepectations,
+enabled by specific macros:
+
+- :c:macro:`Py_USING_SEMI_STABLE_API` exposes API that may change
+ without deprecation warnings.
+- :c:macro:`Py_LIMITED_API` exposes API that is compatible across
+ several minor releases.


exepectations -> expectations
One could also say: "there are ~~two~~public tiers of the API" (emphasis just to show addition); underscored resp. Py_BUILD_CORE are arguably a tier as well...?
I've struck out the "two" because I think it would be clearer to expose the full hierarchy, à la:
Suggested change
There are two tiers of API with different stability exepectations,
enabled by specific macros:
- :c:macro:`Py_USING_SEMI_STABLE_API` exposes API that may change
without deprecation warnings.
- :c:macro:`Py_LIMITED_API` exposes API that is compatible across
several minor releases.
There are three public tiers of the API with different stability
expectations, two of which need to be enabled by specific macros:
- :c:macro:`Py_LIMITED_API` exposes API that is compatible across
several minor releases.
- (no macro): default C-API covered by :pep:`387`.
- :c:macro:`Py_USING_SEMI_STABLE_API` exposes API that may change
without deprecation warnings between minor releases.
I'd also suggest to lead with the most stable one.

By “There are two tiers of API with different stability expectations” I wanted to present them as being additional, different from the default that's covered in the preceding paragraphs.
The order matches the detailed sections below, which would be somewhat tricky to reorder -- the one-paragraph semi-stable tier would feel lost if it was at the end.

Doc/c-api/stable.rst

Doc/c-api/code.rst

h-vetinari · 2022-04-28T03:25:00Z

Doc/c-api/code.rst

+ use :c:func:`PyCode_NewEmpty` instead.
+
+ Since the definition of the bytecode changes often, calling
+:c:func:`PyCode_New` directly can bind you to a precise Python version.


style: better "can directly" than "directly can" (or directly at the very end)

It's not “can directly bind you”, but “calling :c:func:PyCode_New directly” (as opposed to PyCode_Replace for example)

h-vetinari · 2022-04-28T03:27:34Z

Misc/NEWS.d/next/C API/2022-04-21-17-25-22.gh-issue-91744.FgvaMi.rst

@@ -0,0 +1,10 @@
+Defining :c:macro:`Py_USING_SEMI_STABLE_API` will expose "semi-stable API",


Perhaps it's worth considering to stay consistent with the spelling of "semi-stable" / "semistable". Currently the include folders are named like the latter (which I think looks good in a path), mentions in the docs like here often have the hyphen.
Even though the hyphen is grammatically slightly more correct, my personal preference would be without.

Which also opens the question if the macro should be Py_USING_SEMI_STABLE_API or Py_USING_SEMISTABLE_API

I still have hope that someone will come up with a better name. Python-dev is usually so good at bikeshedding!
I'll postpone renaming for now.

encukou · 2022-04-28T14:05:36Z

Thank you for the comments!

vstinner

I expected the API and macro to mention "unstable" rather than "semi-stable" (Py_USING_SEMI_STABLE_API => Py_USING_UNSTABLE_API).

vstinner · 2022-05-04T10:05:34Z

Include/cpython/code.h

-PyObject*, PyObject*, PyObject*, int, PyObject*,
-PyObject*);
-/* same as struct above */
+/* See Include/semi-stable/code.h for PyCode_New* */


I'm not sure that this comment is useful.
Suggested change
/* See Include/semi-stable/code.h for PyCode_New* */
/* See Include/unstable/code.h for PyCode_New* */

vstinner · 2022-05-04T10:06:14Z

Include/pyport.h


+// Expose unstable API when building Python
+#ifdefPy_BUILD_CORE
+#definePy_USING_UNSTABLE_API


Suggested change
#definePy_USING_UNSTABLE_API
#definePy_USING_UNSTABLE_API

vstinner · 2022-05-04T10:06:34Z

Include/pyport.h

+#ifdefPy_USING_UNSTABLE_API
+#define_Py_NEWLY_UNSTABLE(VERSION_UNUSED)
+#else
+#define_Py_NEWLY_UNSTABLE(VERSION) Py_DEPRECATED(VERSION)
+#endif


Suggested change
#ifdefPy_USING_UNSTABLE_API
#define_Py_NEWLY_UNSTABLE(VERSION_UNUSED)
#else
#define_Py_NEWLY_UNSTABLE(VERSION) Py_DEPRECATED(VERSION)
#endif
#ifdefPy_USING_UNSTABLE_API
#define_Py_NEWLY_UNSTABLE(VERSION_UNUSED)
#else
#define_Py_NEWLY_UNSTABLE(VERSION) Py_DEPRECATED(VERSION)
#endif

vstinner · 2022-05-04T10:08:15Z

Include/unstable/code.h

+PyObject*, PyObject*, PyObject*, PyObject*,
+PyObject*, PyObject*, PyObject*, int, PyObject*,
+PyObject*);
+/* same as struct PyCodeObject */


I suggest to move the comment before the function definition: "same parameters as PyCodeObject structure members".

vstinner · 2022-05-04T10:09:20Z

Include/unstable/pystate.h

+
+/* Frame evaluation API (added for PEP-523) */
+
+struct_PyInterpreterFrame;


Is this needed?

vstinner · 2022-05-04T10:13:35Z

Doc/c-api/stable.rst

+
+- :c:macro:`Py_USING_SEMI_STABLE_API` exposes API that may change
+ without deprecation warnings.
+- :c:macro:`Py_LIMITED_API` limits exposed API to API that is compatible


Is API repeated on purpose?
Suggested change
- :c:macro:`Py_LIMITED_API` limits exposed API to API that is compatible
- :c:macro:`Py_LIMITED_API` limits exposed API that is compatible

vstinner · 2022-05-04T10:15:21Z

Include/cpython/code.h


+#definePy_UNSTABLE_CODE_H
+#include"unstable/code.h"
+#undef Py_UNSTABLE_CODE_H


I would prefer to include it in top-level Include/code.h, as done in Include/pystate.h.

vstinner · 2022-05-04T10:18:13Z

Doc/whatsnew/3.11.rst

+ - :c:func:`PyInterpreterState_GetEvalFrameFunc`
+ - :c:func:`PyInterpreterState_SetEvalFrameFunc`
+ - :c:type:`PyFrameEvalFunction`
+


Maybe mention also _PyFrameEvalFunction type which is renamed?

encukou · 2022-08-10T09:55:15Z

I'm deferring the PEP. Thanks for the comments, I'll take them into account next time I try.

encukou added 6 commits April 21, 2022 17:31

Add the semi-stable C-API tier, with PEP-523 functions & PyCode_New*
0185c42

Document Py_USING_SEMI_STABLE_API & functions that need it
381fc11

Add PyFrameEvalFunction & related functions to semi-stable API
23e8a1d

Docs: Expand the intro for C API stability
02c5466

Add blurb
e8e1884

Add What's New entry
a441699

encukou requested review from ncoghlan and vstinner April 21, 2022 15:56

encukou requested a review from markshannon as a code owner April 21, 2022 15:56

bedevere-bot added the awaiting core review label Apr 21, 2022

encukou added 2 commits April 21, 2022 17:59

Merge main branch to resolve a conflict
31beafd

And one more fix for the conflict
132fd8e

encukou marked this pull request as draft April 21, 2022 16:07

encukou mentioned this pull request Apr 21, 2022
PEP 689 -- Add an unstable C-API tier #91744
Closed
19 tasks

Exclude semistable/ from the limited API
7dba38c
d'oh!

vstinner reviewed Apr 21, 2022
View reviewed changes

vstinner mentioned this pull request Apr 26, 2022
gh-91719: Add pycore_opcode.h internal header file #91906
Merged

h-vetinari reviewed Apr 28, 2022
View reviewed changes

Address some review suggestions
74419bd

encukou added 2 commits April 28, 2022 16:13

Merge main branch to resolve conflicts
31ea0db

Fix issue reference in docs
75abb20

vstinner mentioned this pull request May 2, 2022
[C API] Move the PyCodeObject structure to the internal C API (make the structure opaque) #91397
Closed

encukou added 2 commits May 4, 2022 01:16

Rename semi-stable to unstable
b189c81

Merge main branch to resolve conflicts
911f78a

vstinner reviewed May 4, 2022
View reviewed changes

encukou closed this Aug 10, 2022

		@@ -0,0 +1,10 @@
		Defining :c:macro:`Py_USING_SEMI_STABLE_API` will expose "semi-stable API",

	/* See Include/semi-stable/code.h for PyCode_New* */
	/* See Include/unstable/code.h for PyCode_New* */


		/* Frame evaluation API (added for PEP-523) */

		struct_PyInterpreterFrame;

	- :c:macro:`Py_LIMITED_API` limits exposed API to API that is compatible
	- :c:macro:`Py_LIMITED_API` limits exposed API that is compatible

Uh oh!

gh-91744: Add semi-stable C API tier#91789

gh-91744: Add semi-stable C API tier #91789

Uh oh!

Conversation

encukou commented Apr 21, 2022

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

vstinner commented Apr 21, 2022

Uh oh!

vstinner commented Apr 26, 2022

Uh oh!

h-vetinari left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

encukouApr 28, 2022• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

encukou commented Apr 28, 2022

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

encukou commented Aug 10, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

encukouApr 28, 2022•
edited
Loading