Skip to content

gh-77273: Better bytecodes for f-strings#6132

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
markshannon merged 6 commits into from
Jun 14, 2023
Merged

gh-77273: Better bytecodes for f-strings #6132

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
3af4084
Avoid conditional stack effects when compiling f-strings.
markshannon Jun 5, 2023
a84bd65
Merge branch 'main' into better-bytecode-fstring
markshannon Jun 5, 2023
b6d7171
Fix up magic number.
markshannon Jun 5, 2023
62a3404
Document new instructions.
markshannon Jun 5, 2023
31385d1
Merge branch 'main' into better-bytecode-fstring
markshannon Jun 9, 2023
641466c
Merge branch 'main' into better-bytecode-fstring
markshannon Jun 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 37 additions & 16 deletions Doc/library/dis.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1465,26 +1465,47 @@ iterations of the loop.
an argument from two-byte to four-byte.


.. opcode:: FORMAT_VALUE (flags)
.. opcode:: CONVERT_VALUE (oparg)

Used for implementing formatted literal strings (f-strings). Pops
an optional *fmt_spec* from the stack, then a required *value*.
*flags* is interpreted as follows:
Convert value to a string, depending on ``oparg``::

* ``(flags & 0x03) == 0x00``: *value* is formatted as-is.
* ``(flags & 0x03) == 0x01``: call :func:`str` on *value* before
formatting it.
* ``(flags & 0x03) == 0x02``: call :func:`repr` on *value* before
formatting it.
* ``(flags & 0x03) == 0x03``: call :func:`ascii` on *value* before
formatting it.
* ``(flags & 0x04) == 0x04``: pop *fmt_spec* from the stack and use
it, else use an empty *fmt_spec*.
value = STACK.pop()
result = func(value)
STACK.push(result)

Formatting is performed using :c:func:`PyObject_Format`. The
result is pushed on the stack.
* ``oparg == 1``: call :func:`str` on *value*
* ``oparg == 2``: call :func:`repr` on *value*
* ``oparg == 3``: call :func:`ascii` on *value*

.. versionadded:: 3.6
Used for implementing formatted literal strings (f-strings).

.. versionadded:: 3.13


.. opcode:: FORMAT_SIMPLE

Formats the value on top of stack::

value = STACK.pop()
result = value.__format__("")
STACK.push(result)

Used for implementing formatted literal strings (f-strings).

.. versionadded:: 3.13

.. opcode:: FORMAT_SPEC

Formats the given value with the given format spec::

spec = STACK.pop()
value = STACK.pop()
result = value.__format__(spec)
STACK.push(result)

Used for implementing formatted literal strings (f-strings).

.. versionadded:: 3.13


.. opcode:: MATCH_CLASS (count)
Expand Down
50 changes: 25 additions & 25 deletions Include/internal/pycore_opcode.h
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

86 changes: 44 additions & 42 deletions Include/opcode.h
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 5 additions & 14 deletions Lib/dis.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -25,13 +25,8 @@
_have_code = (types.MethodType, types.FunctionType, types.CodeType,
classmethod, staticmethod, type)

FORMAT_VALUE = opmap['FORMAT_VALUE']
FORMAT_VALUE_CONVERTERS = (
(None, ''),
(str, 'str'),
(repr, 'repr'),
(ascii, 'ascii'),
)
CONVERT_VALUE = opmap['CONVERT_VALUE']

SET_FUNCTION_ATTRIBUTE = opmap['SET_FUNCTION_ATTRIBUTE']
FUNCTION_ATTR_FLAGS = ('defaults', 'kwdefaults', 'annotations', 'closure')

Expand DownExpand Up@@ -579,13 +574,9 @@ def _get_instructions_bytes(code, varname_from_oparg=None,
elif deop in hascompare:
argval = cmp_op[arg>>4]
argrepr = argval
elif deop == FORMAT_VALUE:
argval, argrepr = FORMAT_VALUE_CONVERTERS[arg & 0x3]
argval = (argval, bool(arg & 0x4))
if argval[1]:
if argrepr:
argrepr += ', '
argrepr += 'with format'
elif deop == CONVERT_VALUE:
argval = (None, str, repr, ascii)[arg]
argrepr = ('', 'str', 'repr', 'ascii')[arg]
Comment on lines +578 to +579
Copy link
Member

@iritkatrieliritkatrielJun 8, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since oparg 0 isn't a thing (right?), maybe write this as:

Suggested change
argval= (None, str, repr, ascii)[arg]
argrepr= ('', 'str', 'repr', 'ascii')[arg]
argval= (_, str, repr, ascii)[arg]
argrepr= (_, 'str', 'repr', 'ascii')[arg]

Copy link
MemberAuthor

@markshannonmarkshannonJun 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is _? We might not use the 0th item, but it still needs to be defined.

Copy link
Member

@iritkatrieliritkatrielJun 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC _ is the convention for ‘unused value’.

Copy link
MemberAuthor

@markshannonmarkshannonJun 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For lvalues, sure, but this is an rvalue.

elif deop == SET_FUNCTION_ATTRIBUTE:
argrepr = ', '.join(s for i, s in enumerate(FUNCTION_ATTR_FLAGS)
if arg & (1<<i))
Expand Down
6 changes: 4 additions & 2 deletions Lib/importlib/_bootstrap_external.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -447,7 +447,9 @@ def _write_atomic(path, data, mode=0o666):
# Python 3.12b1 3531 (Add PEP 695 changes)
# Python 3.13a1 3550 (Plugin optimizer support)
# Python 3.13a1 3551 (Compact superinstructions)
# Python 3.13a1 3552 (Add SET_FUNCTION_ATTRIBUTE)
# Python 3.13a1 3552 (Remove LOAD_FAST__LOAD_CONST and LOAD_CONST__LOAD_FAST)
# Python 3.13a1 3553 (Add SET_FUNCTION_ATTRIBUTE)
# Python 3.13a1 3554 (more efficient bytecodes for f-strings)

# Python 3.14 will start with 3600

Expand All@@ -464,7 +466,7 @@ def _write_atomic(path, data, mode=0o666):
# Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array
# in PC/launcher.c must also be updated.

MAGIC_NUMBER = (3552).to_bytes(2, 'little') + b'\r\n'
MAGIC_NUMBER = (3554).to_bytes(2, 'little') + b'\r\n'

_RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c

Expand Down
5 changes: 4 additions & 1 deletion Lib/opcode.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -110,6 +110,9 @@ def pseudo_op(name, op, real_ops):
def_op('CHECK_EXC_MATCH', 36)
def_op('CHECK_EG_MATCH', 37)

def_op('FORMAT_SIMPLE', 40)
def_op('FORMAT_WITH_SPEC', 41)

def_op('WITH_EXCEPT_START', 49)
def_op('GET_AITER', 50)
def_op('GET_ANEXT', 51)
Expand DownExpand Up@@ -213,9 +216,9 @@ def pseudo_op(name, op, real_ops):
def_op('RESUME', 151) # This must be kept in sync with deepfreeze.py
def_op('MATCH_CLASS', 152)

def_op('FORMAT_VALUE', 155)
def_op('BUILD_CONST_KEY_MAP', 156)
def_op('BUILD_STRING', 157)
def_op('CONVERT_VALUE', 158)

def_op('LIST_EXTEND', 162)
def_op('SET_UPDATE', 163)
Expand Down
Loading