Skip to content

bpo-45292: [PEP-654] add except*#29581

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
iritkatriel merged 50 commits into from
Dec 14, 2021
Merged

bpo-45292: [PEP-654] add except* #29581

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
b0ca0b7
bpo-45292: [PEP-654] add except*
iritkatriel Nov 11, 2021
cad9126
fold invalid_except_star_stmt into invalid_except_stmt. Add custom me…
iritkatriel Nov 17, 2021
798aaa7
Erlend's first review
iritkatriel Nov 17, 2021
f95272c
fix grammar rule for mixed except/except*
iritkatriel Nov 18, 2021
ca44878
fix doctest failure
iritkatriel Nov 18, 2021
adc834d
remove duplication in except* handling
iritkatriel Nov 19, 2021
e3fa2f7
fix bug - move POP_BLOCK to the end in case of except*
iritkatriel Nov 19, 2021
1908fd8
Merge remote-tracking branch 'upstream/main' into bpo-45292-except_star
iritkatriel Nov 21, 2021
5846329
Merge remote-tracking branch 'upstream/main' into bpo-45292-except_star
iritkatriel Nov 21, 2021
f9a626c
Use Py_NewRef(_PyLong_GetZero());
iritkatriel Nov 21, 2021
fdc948b
Split AST node Try -> Try/TryStar. Separate try-except and try-except…
iritkatriel Nov 22, 2021
488f9fa
Revert doctest changes (not needed with TryStar)
iritkatriel Nov 22, 2021
4eff693
a pesky little decref
iritkatriel Nov 22, 2021
211d6da
📜🤖 Added by blurb_it.
blurb-it[bot] Nov 22, 2021
03dbf43
update unparse for TryStar
iritkatriel Nov 22, 2021
704bc8f
update ast doc with TryStar
iritkatriel Nov 22, 2021
ff36e5d
add try-star test in test_grammar
iritkatriel Nov 22, 2021
73ee294
add test for invalid exception matcher (old except)
iritkatriel Nov 22, 2021
51e8178
break/ continue/return in except* block is SyntaxError
iritkatriel Nov 22, 2021
50a2126
add stack stability tests for except*
iritkatriel Nov 22, 2021
1d9969d
Merge remote-tracking branch 'upstream/main' into bpo-45292-except_star
iritkatriel Nov 22, 2021
4a7bf9f
remove commented out debug prints
iritkatriel Nov 23, 2021
3ad5d41
add trace tests for except*
iritkatriel Nov 24, 2021
2813363
remove obsolete comment
iritkatriel Nov 24, 2021
7e54805
formatting of comment
iritkatriel Nov 25, 2021
5736f15
save step before try
iritkatriel Nov 25, 2021
82a4db3
add diagram of stack contents durign except*
iritkatriel Nov 25, 2021
a461e7d
one more save step before try
iritkatriel Nov 25, 2021
be609b3
fix diagram
iritkatriel Nov 25, 2021
706e54b
fix diagram some more
iritkatriel Nov 25, 2021
7f49575
treat JUMP_IF_NOT_EG_MATCH like *_EXC_MATCH in mark_stacks
iritkatriel Nov 27, 2021
ae729f3
whatnew in 3.11
iritkatriel Nov 27, 2021
38e3b50
Merge branch 'main' into bpo-45292-except_star
ambv Nov 30, 2021
ee53c8d
Merge remote-tracking branch 'upstream/main' into bpo-45292-except_star
iritkatriel Dec 5, 2021
9c18862
Merge remote-tracking branch 'upstream/main' into bpo-45292-except_star
iritkatriel Dec 7, 2021
7428b15
Merge remote-tracking branch 'upstream/main' into bpo-45292-except_star
iritkatriel Dec 8, 2021
bbe44fb
fix typos in dis doc entry
iritkatriel Dec 8, 2021
22bf207
"three items" --> "3items"
iritkatriel Dec 8, 2021
66d830a
add spaces
iritkatriel Dec 8, 2021
f2b74a5
Merge remote-tracking branch 'upstream/main' into bpo-45292-except_star
iritkatriel Dec 10, 2021
035fe5c
Apply first batch of suggestions from code review
iritkatriel Dec 11, 2021
5d52de3
the remaining comments from Erlend
iritkatriel Dec 11, 2021
a1591f0
sign what's new
iritkatriel Dec 11, 2021
ce36211
Merge remote-tracking branch 'upstream/main' into bpo-45292-except_star
iritkatriel Dec 11, 2021
20d9ccd
'== -1' --> '< 0' for consistency. Added a missing recursion guard.
iritkatriel Dec 12, 2021
3a642e6
Merge remote-tracking branch 'upstream/main' into bpo-45292-except_star
iritkatriel Dec 12, 2021
13e0d0f
Merge branch 'main' into bpo-45292-except_star
iritkatriel Dec 13, 2021
078f050
whitespace
iritkatriel Dec 14, 2021
a91b219
Merge branch 'bpo-45292-except_star' of https://github.com/iritkatrie…
iritkatriel Dec 14, 2021
564b8a0
Merge remote-tracking branch 'upstream/main' into bpo-45292-except_star
iritkatriel Dec 14, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions Doc/library/ast.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1167,6 +1167,37 @@ Control flow
type_ignores=[])


.. class:: TryStar(body, handlers, orelse, finalbody)

``try`` blocks which are followed by ``except*`` clauses. The attributes are the
same as for :class:`Try` but the :class:`ExceptHandler` nodes in ``handlers``
are interpreted as ``except*`` blocks rather then ``except``.

.. doctest::

>>> print(ast.dump(ast.parse("""
... try:
... ...
... except* Exception:
... ...
... """), indent=4))
Module(
body=[
TryStar(
body=[
Expr(
value=Constant(value=Ellipsis))],
handlers=[
ExceptHandler(
type=Name(id='Exception', ctx=Load()),
body=[
Expr(
value=Constant(value=Ellipsis))])],
orelse=[],
finalbody=[])],
type_ignores=[])


.. class:: ExceptHandler(type, name, body)

A single ``except`` clause. ``type`` is the exception type it will match,
Expand Down
26 changes: 26 additions & 0 deletions Doc/library/dis.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -872,8 +872,10 @@ All of the following opcodes use their arguments.

.. versionadded:: 3.1


.. opcode:: JUMP_IF_NOT_EXC_MATCH (target)

Performs exception matching for ``except``.
Tests whether the second value on the stack is an exception matching TOS,
and jumps if it is not. Pops one value from the stack.

Expand All@@ -883,6 +885,30 @@ All of the following opcodes use their arguments.
This opcode no longer pops the active exception.


.. opcode:: JUMP_IF_NOT_EG_MATCH (target)

Performs exception matching for ``except*``. Applies ``split(TOS)`` on
the exception group representing TOS1. Jumps if no match is found.

Pops one item from the stack. If a match was found, pops the 3 items representing
the exception and pushes the 3 items representing the non-matching part of
the exception group, followed by the 3 items representing the matching part.
In other words, in case of a match it pops 4 items and pushes 6.

.. versionadded:: 3.11


.. opcode:: PREP_RERAISE_STAR

Combines the raised and reraised exceptions list from TOS, into an exception
group to propagate from a try-except* block. Uses the original exception
group from TOS1 to reconstruct the structure of reraised exceptions. Pops
two items from the stack and pushes a triplet representing the exception to
reraise or three ``None`` if there isn't one.

.. versionadded:: 3.11


.. opcode:: JUMP_IF_TRUE_OR_POP (target)

If TOS is true, sets the bytecode counter to *target* and leaves TOS on the
Expand Down
2 changes: 2 additions & 0 deletions Doc/whatsnew/3.11.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -65,6 +65,8 @@ Summary -- Release highlights

.. PEP-sized items next.

PEP-654: Exception Groups and ``except*``.
(Contributed by Irit Katriel in :issue:`45292`.)

New Features
============
Expand Down
17 changes: 15 additions & 2 deletions Grammar/python.gram
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -403,6 +403,8 @@ try_stmt[stmt_ty]:
| invalid_try_stmt
| 'try' &&':' b=block f=finally_block{_PyAST_Try(b, NULL, NULL, f, EXTRA) }
| 'try' &&':' b=block ex[asdl_excepthandler_seq*]=except_block+ el=[else_block] f=[finally_block]{_PyAST_Try(b, ex, el, f, EXTRA) }
| 'try' &&':' b=block ex[asdl_excepthandler_seq*]=except_star_block+ el=[else_block] f=[finally_block]{_PyAST_TryStar(b, ex, el, f, EXTRA) }


# Except statement
# ----------------
Expand All@@ -413,6 +415,11 @@ except_block[excepthandler_ty]:
_PyAST_ExceptHandler(e, (t) ? ((expr_ty) t)->v.Name.id : NULL, b, EXTRA) }
| 'except' ':' b=block{_PyAST_ExceptHandler(NULL, NULL, b, EXTRA) }
| invalid_except_stmt
except_star_block[excepthandler_ty]:
| invalid_except_star_stmt_indent
| 'except' '*' e=expression t=['as' z=NAME{z }] ':' b=block{
_PyAST_ExceptHandler(e, (t) ? ((expr_ty) t)->v.Name.id : NULL, b, EXTRA) }
| invalid_except_stmt
finally_block[asdl_stmt_seq*]:
| invalid_finally_stmt
| 'finally' &&':' a=block{a }
Expand DownExpand Up@@ -1192,18 +1199,24 @@ invalid_try_stmt:
| a='try' ':' NEWLINE !INDENT{
RAISE_INDENTATION_ERROR("expected an indented block after 'try' statement on line %d", a->lineno) }
| 'try' ':' block !('except' | 'finally'){RAISE_SYNTAX_ERROR("expected 'except' or 'finally' block") }
| 'try' ':' block* ((except_block+ except_star_block) | (except_star_block+ except_block)) block*{
RAISE_SYNTAX_ERROR("cannot have both 'except' and 'except*' on the same 'try'") }
invalid_except_stmt:
| 'except' a=expression ',' expressions ['as' NAME ] ':'{
| 'except' '*'? a=expression ',' expressions ['as' NAME ] ':'{
RAISE_SYNTAX_ERROR_STARTING_FROM(a, "multiple exception types must be parenthesized") }
| a='except' expression ['as' NAME ] NEWLINE{RAISE_SYNTAX_ERROR("expected ':'") }
| a='except' '*'? expression ['as' NAME ] NEWLINE{RAISE_SYNTAX_ERROR("expected ':'") }
| a='except' NEWLINE{RAISE_SYNTAX_ERROR("expected ':'") }
| a='except' '*' (NEWLINE | ':'){RAISE_SYNTAX_ERROR("expected one or more exception types") }
invalid_finally_stmt:
| a='finally' ':' NEWLINE !INDENT{
RAISE_INDENTATION_ERROR("expected an indented block after 'finally' statement on line %d", a->lineno) }
invalid_except_stmt_indent:
| a='except' expression ['as' NAME ] ':' NEWLINE !INDENT{
RAISE_INDENTATION_ERROR("expected an indented block after 'except' statement on line %d", a->lineno) }
| a='except' ':' NEWLINE !INDENT{RAISE_SYNTAX_ERROR("expected an indented block after except statement on line %d", a->lineno) }
invalid_except_star_stmt_indent:
| a='except' '*' expression ['as' NAME ] ':' NEWLINE !INDENT{
RAISE_INDENTATION_ERROR("expected an indented block after 'except*' statement on line %d", a->lineno) }
invalid_match_stmt:
| "match" subject_expr !':'{CHECK_VERSION(void*, 10, "Pattern matching is", RAISE_SYNTAX_ERROR("expected ':'") ) }
| a="match" subject=subject_expr ':' NEWLINE !INDENT{
Expand Down
17 changes: 14 additions & 3 deletions Include/internal/pycore_ast.h
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Include/internal/pycore_ast_state.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -132,6 +132,7 @@ struct ast_state{
PyObject *Sub_singleton;
PyObject *Sub_type;
PyObject *Subscript_type;
PyObject *TryStar_type;
PyObject *Try_type;
PyObject *Tuple_type;
PyObject *TypeIgnore_type;
Expand Down
8 changes: 8 additions & 0 deletions Include/internal/pycore_pyerrors.h
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -92,6 +92,14 @@ PyAPI_FUNC(PyObject *) _PyErr_FormatFromCauseTstate(
const char *format,
...);

PyAPI_FUNC(PyObject *) _PyExc_CreateExceptionGroup(
const char *msg,
PyObject *excs);

PyAPI_FUNC(PyObject *) _PyExc_ExceptionGroupProjection(
PyObject *left,
PyObject *right);

PyAPI_FUNC(int) _PyErr_CheckSignalsTstate(PyThreadState *tstate);

PyAPI_FUNC(void) _Py_DumpExtensionModules(int fd, PyInterpreterState *interp);
Expand Down
6 changes: 4 additions & 2 deletions Include/opcode.h
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 19 additions & 2 deletions Lib/ast.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -683,6 +683,7 @@ def __init__(self, *, _avoid_backslashes=False):
self._type_ignores ={}
self._indent = 0
self._avoid_backslashes = _avoid_backslashes
self._in_try_star = False

def interleave(self, inter, f, seq):
"""Call f on each item in seq, calling inter() in between."""
Expand DownExpand Up@@ -953,7 +954,7 @@ def visit_Raise(self, node):
self.write(" from ")
self.traverse(node.cause)

def visit_Try(self, node):
def do_visit_try(self, node):
self.fill("try")
with self.block():
self.traverse(node.body)
Expand All@@ -968,8 +969,24 @@ def visit_Try(self, node):
with self.block():
self.traverse(node.finalbody)

def visit_Try(self, node):
prev_in_try_star = self._in_try_star
try:
self._in_try_star = False
self.do_visit_try(node)
finally:
self._in_try_star = prev_in_try_star

def visit_TryStar(self, node):
prev_in_try_star = self._in_try_star
try:
self._in_try_star = True
self.do_visit_try(node)
finally:
self._in_try_star = prev_in_try_star

def visit_ExceptHandler(self, node):
self.fill("except")
self.fill("except*" if self._in_try_star else "except")
if node.type:
self.write(" ")
self.traverse(node.type)
Expand Down
3 changes: 2 additions & 1 deletion Lib/importlib/_bootstrap_external.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -371,6 +371,7 @@ def _write_atomic(path, data, mode=0o666):
# Python 3.11a3 3464 (bpo-45636: Merge numeric BINARY_*/INPLACE_* into
# BINARY_OP)
# Python 3.11a3 3465 (Add COPY_FREE_VARS opcode)
# Python 3.11a3 3466 (bpo-45292: PEP-654 except*)

#
# MAGIC must change whenever the bytecode emitted by the compiler may no
Expand All@@ -380,7 +381,7 @@ def _write_atomic(path, data, mode=0o666):
# Whenever MAGIC_NUMBER is changed, the ranges in the magic_values array
# in PC/launcher.c must also be updated.

MAGIC_NUMBER = (3465).to_bytes(2, 'little') + b'\r\n'
MAGIC_NUMBER = (3466).to_bytes(2, 'little') + b'\r\n'
_RAW_MAGIC_NUMBER = int.from_bytes(MAGIC_NUMBER, 'little') # For import.c

_PYCACHE = '__pycache__'
Expand Down
3 changes: 3 additions & 0 deletions Lib/opcode.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -103,6 +103,7 @@ def jabs_op(name, op):
def_op('SETUP_ANNOTATIONS', 85)
def_op('YIELD_VALUE', 86)

def_op('PREP_RERAISE_STAR', 88)
def_op('POP_EXCEPT', 89)

HAVE_ARGUMENT = 90 # Opcodes from here have an argument:
Expand DownExpand Up@@ -150,6 +151,8 @@ def jabs_op(name, op):
def_op('DELETE_FAST', 126) # Local variable number
haslocal.append(126)

jabs_op('JUMP_IF_NOT_EG_MATCH', 127)

def_op('GEN_START', 129) # Kind of generator/coroutine
def_op('RAISE_VARARGS', 130) # Number of raise arguments (1, 2, or 3)
def_op('CALL_FUNCTION', 131) # #args
Expand Down
23 changes: 23 additions & 0 deletions Lib/test/test_ast.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -86,6 +86,8 @@ def to_tuple(t):
"try:\n pass\nexcept Exception:\n pass",
# TryFinally
"try:\n pass\nfinally:\n pass",
# TryStarExcept
"try:\n pass\nexcept* Exception:\n pass",
# Assert
"assert v",
# Import
Expand DownExpand Up@@ -1310,6 +1312,26 @@ def test_try(self):
t = ast.Try([p], e, [p], [ast.Expr(ast.Name("x", ast.Store()))])
self.stmt(t, "must have Load context")

def test_try_star(self):
p = ast.Pass()
t = ast.TryStar([], [], [], [p])
self.stmt(t, "empty body on TryStar")
t = ast.TryStar([ast.Expr(ast.Name("x", ast.Store()))], [], [], [p])
self.stmt(t, "must have Load context")
t = ast.TryStar([p], [], [], [])
self.stmt(t, "TryStar has neither except handlers nor finalbody")
t = ast.TryStar([p], [], [p], [p])
self.stmt(t, "TryStar has orelse but no except handlers")
t = ast.TryStar([p], [ast.ExceptHandler(None, "x", [])], [], [])
self.stmt(t, "empty body on ExceptHandler")
e = [ast.ExceptHandler(ast.Name("x", ast.Store()), "y", [p])]
self.stmt(ast.TryStar([p], e, [], []), "must have Load context")
e = [ast.ExceptHandler(None, "x", [p])]
t = ast.TryStar([p], e, [ast.Expr(ast.Name("x", ast.Store()))], [p])
self.stmt(t, "must have Load context")
t = ast.TryStar([p], e, [p], [ast.Expr(ast.Name("x", ast.Store()))])
self.stmt(t, "must have Load context")

def test_assert(self):
self.stmt(ast.Assert(ast.Name("x", ast.Store()), None),
"must have Load context")
Expand DownExpand Up@@ -2316,6 +2338,7 @@ def main():
('Module', [('Raise', (1, 0, 1, 25), ('Call', (1, 6, 1, 25), ('Name', (1, 6, 1, 15), 'Exception', ('Load',)), [('Constant', (1, 16, 1, 24), 'string', None)], []), None)], []),
('Module', [('Try', (1, 0, 4, 6), [('Pass', (2, 2, 2, 6))], [('ExceptHandler', (3, 0, 4, 6), ('Name', (3, 7, 3, 16), 'Exception', ('Load',)), None, [('Pass', (4, 2, 4, 6))])], [], [])], []),
('Module', [('Try', (1, 0, 4, 6), [('Pass', (2, 2, 2, 6))], [], [], [('Pass', (4, 2, 4, 6))])], []),
('Module', [('TryStar', (1, 0, 4, 6), [('Pass', (2, 2, 2, 6))], [('ExceptHandler', (3, 0, 4, 6), ('Name', (3, 8, 3, 17), 'Exception', ('Load',)), None, [('Pass', (4, 2, 4, 6))])], [], [])], []),
('Module', [('Assert', (1, 0, 1, 8), ('Name', (1, 7, 1, 8), 'v', ('Load',)), None)], []),
('Module', [('Import', (1, 0, 1, 10), [('alias', (1, 7, 1, 10), 'sys', None)])], []),
('Module', [('ImportFrom', (1, 0, 1, 17), 'sys', [('alias', (1, 16, 1, 17), 'v', None)], 0)], []),
Expand Down
33 changes: 33 additions & 0 deletions Lib/test/test_compile.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -1263,6 +1263,39 @@ def test_try_except_as(self):
"""
self.check_stack_size(snippet)

def test_try_except_star_qualified(self):
snippet = """
try:
a
except* ImportError:
b
else:
c
"""
self.check_stack_size(snippet)

def test_try_except_star_as(self):
snippet = """
try:
a
except* ImportError as e:
b
else:
c
"""
self.check_stack_size(snippet)

def test_try_except_star_finally(self):
snippet = """
try:
a
except* A:
b
finally:
c
"""
self.check_stack_size(snippet)

def test_try_finally(self):
snippet = """
try:
Expand Down
Loading