[subinterpreters] Get the current Python interpreter state from Thread Local Storage (autoTSSkey)

[vstinner]

BPO	40522
Nosy	@vstinner, @seberg, @corona10, @pablogsal, @shihai1991, @h-vetinari
PRs	bpo-40522: _PyThreadState_Swap() sets autoTSSkey #19939 bpo-40522: Store tstate in a Thread Local Storage #23976 bpo-40522: Replace PyThreadState_GET() with PyThreadState_Get() #24575 bpo-40522: Use PyThreadState_Get() in _lsprof.c #24596 bpo-40522: Use PyThreadState_Get() in decimal #24597

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-05-05.17:12:14.636>
labels = ['expert-subinterpreters', '3.9']
title = '[subinterpreters] Get the current Python interpreter state from Thread Local Storage (autoTSSkey)'
updated_at = <Date 2021-06-29.17:34:08.113>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2021-06-29.17:34:08.113>
actor = 'h-vetinari'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Subinterpreters']
creation = <Date 2020-05-05.17:12:14.636>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 40522
keywords = ['patch']
message_count = 11.0
messages = ['368182', '368188', '380324', '383894', '383895', '383899', '383900', '383940', '383967', '384057', '387312']
nosy_count = 6.0
nosy_names = ['vstinner', 'seberg', 'corona10', 'pablogsal', 'shihai1991', 'h-vetinari']
pr_nums = ['19939', '23976', '24575', '24596', '24597']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue40522'
versions = ['Python 3.9']

[vstinner]

_PyThreadState_GET() gets the current Python thread state from _PyRuntime.gilstate.tstate_current atomic variable.

When I experimented per-interpreter GIL (bpo-40512), I got issues with _PyThreadState_GET() which didn't return the expected Python thread state.

I propose to modify _PyThreadState_GET() in the exprimental isolated subinterpreters mode to get and set the current Python thread state using a Thread Local Storage: autoTSSkey.

[vstinner]

New changeset e838a93 by Victor Stinner in branch 'master':
bpo-40522: _PyThreadState_Swap() sets autoTSSkey (GH-19939)
e838a93

[vstinner]

See also bpo-15751: "Make the PyGILState API compatible with subinterpreters".

[vstinner]

There are many ways to get the current interpreter (interp) and the current Python thread state (tstate).

Public C API, opaque function call:

• PyInterpreterState_Get() (new in Python 3.9)
• PyThreadState_Get()

Internal C API, static inline functions:

• _PyInterpreterState_GET()
• _PyThreadState_GET()

There are so many variants that I wrote notes for myself:
https://pythondev.readthedocs.io/pystate.html

This issue is about optimizing _PyInterpreterState_GET() and _PyThreadState_GET() which are supposed to be the most efficient implementations.

Currently, _PyInterpreterState_GET() is implemented as _PyThreadState_GET()->interp, and _PyThreadState_GET() is implemented as:

 _Py_atomic_load_relaxed(_PyRuntime.gilstate.tstate_current)

--

To find the _PyInterpreterState_GET() machine code, I read the PyInterpreterState_Get() assembly code (not optimized, it adds tstate==NULL test) and PyTuple_New() assembly code, since PyTuple_New() now needs to get the current interpreter:

static struct _Py_tuple_state *
get_tuple_state(void)
{
 PyInterpreterState *interp = _PyInterpreterState_GET();
 return &interp->tuple;
}

To find the _PyThreadState_GET() machine code, I read the PyThreadState_Get() assembly code.

I looked at the x86-64 machine code generated by GCC -O3 (no LTO, no PGO, it should not be relevant here), using GCC 10.2.1 on Fedora 33.

_PyThreadState_GET():

mov rax,QWORD PTR [rip+0x2292b1] # 0x743118 <_PyRuntime+568>

_PyInterpreterState_GET():

mov rax,QWORD PTR [rip+0x22a7dd] # 0x743118 <_PyRuntime+568>
mov rax,QWORD PTR [rax+0x10]

By default, Python is built with -fPIC: _PyRuntime variable does not have a fixed address.

$ objdump -t ./python|grep '\<_PyRuntime\>'
0000000000742ee0 g O .bss	00000000000002a0 _PyRuntime

The "[rip+0x2292b1] # 0x743118 <_PyRuntime+568>" indirection is needed by PIC.

[vstinner]

_PyInterpreterState_GET():
mov rax,QWORD PTR [rip+0x22a7dd] # 0x743118 <_PyRuntime+568>
mov rax,QWORD PTR [rax+0x10]

While working on bpo-39465, I wrote PR 20767 to optimize _PyInterpreterState_GET(): single instruction instead of two:

• Add _PyRuntimeState.interp_current member: atomic variable
• _PyThreadState_Swap() sets _PyRuntimeState.interp_current

But I failed to measure any performance difference.