line number for a return in a with block after 3.10 is that of the with, not the return statement

[gpshead]

def foo() -> int:
 with open('/dev/null') as devnull:
 return 'a'

In Python 3.10 onwards, the reported line number for the return statement is incorrect.

Python main 3.12-main-ish not far from 3.11:

>>> def foo() -> int:
... with open('a') as f:
... return 'b'
... 
>>> dis.dis(foo)
 1 0 RESUME 0

 2 2 LOAD_GLOBAL 1 (NULL + open)
 14 LOAD_CONST 1 ('a')
 16 CALL 1
 26 BEFORE_WITH
 28 STORE_FAST 0 (f)

 3 30 NOP

 2 32 LOAD_CONST 0 (None)
 34 LOAD_CONST 0 (None)
 36 LOAD_CONST 0 (None)
 38 CALL 2
 48 POP_TOP
 50 LOAD_CONST 2 ('b')
 52 RETURN_VALUE
 >> 54 PUSH_EXC_INFO
 56 WITH_EXCEPT_START
 58 POP_JUMP_FORWARD_IF_TRUE 4 (to 68)
 60 RERAISE 2
 >> 62 COPY 3
 64 POP_EXCEPT
 66 RERAISE 1
 >> 68 POP_TOP
 70 POP_EXCEPT
 72 POP_TOP
 74 POP_TOP
 76 LOAD_CONST 0 (None)
 78 RETURN_VALUE
ExceptionTable:
 28 to 30 -> 54 [1] lasti
 54 to 60 -> 62 [3] lasti
 68 to 68 -> 62 [3] lasti
>>> sys.version_info
sys.version_info(major=3, minor=12, micro=0, releaselevel='alpha', serial=0)

The only thing attributed to line 3 is a NOP. The RETURN_VALUE of 'b' is listed as line 2. That is wrong.

Python 3.9:

>>> def foo() -> int:
... with open('a') as f:
... return 'b'
... 
>>> dis.dis(foo)
 2 0 LOAD_GLOBAL 0 (open)
 2 LOAD_CONST 1 ('a')
 4 CALL_FUNCTION 1
 6 SETUP_WITH 18 (to 26)
 8 STORE_FAST 0 (f)

 3 10 POP_BLOCK
 12 LOAD_CONST 0 (None)
 14 DUP_TOP
 16 DUP_TOP
 18 CALL_FUNCTION 3
 20 POP_TOP
 22 LOAD_CONST 2 ('b')
 24 RETURN_VALUE
 >> 26 WITH_EXCEPT_START
 28 POP_JUMP_IF_TRUE 32
 30 RERAISE
 >> 32 POP_TOP
 34 POP_TOP
 36 POP_TOP
 38 POP_EXCEPT
 40 POP_TOP
 42 LOAD_CONST 0 (None)
 44 RETURN_VALUE
>>> sys.version_info
sys.version_info(major=3, minor=9, micro=12, releaselevel='final', serial=0)

Noticed by @martindemello working on a 3.10 error attribution bug in pytype.

[dignissimus]

The issue was introduced in 937cebc
See bpo-44298: #88464

The same behaviour occurs with try-finally blocks on both 3.9 and 3.10+

def function():
 try:
 return
 finally:
 pass

With the above code, disassembling on Python 3.10.5 produces the following

 1 0 LOAD_CONST 0 (<code object function at 0x7f96b4e0d000, file "a.py", line 1>)
 2 LOAD_CONST 1 ('function')
 4 MAKE_FUNCTION 0
 6 STORE_NAME 0 (function)
 8 LOAD_CONST 2 (None)
 10 RETURN_VALUE

Disassembly of <code object function at 0x7f96b4e0d000, file "tryfinally.py", line 1>:
 2 0 SETUP_FINALLY 3 (to 8)

 3 2 POP_BLOCK

 5 4 LOAD_CONST 0 (None)
 6 RETURN_VALUE
 >> 8 RERAISE 0

And on 3.9.0

 1 0 LOAD_CONST 0 (<code object function at 0x7f72158a8be0, file "/tmp/a.py", line 1>)
 2 LOAD_CONST 1 ('function')
 4 MAKE_FUNCTION 0
 6 STORE_NAME 0 (function)
 8 LOAD_CONST 2 (None)
 10 RETURN_VALUE

Disassembly of <code object function at 0x7f72158a8be0, file "tryfinally.py", line 1>:
 2 0 SETUP_FINALLY 6 (to 8)

 3 2 POP_BLOCK
 4 LOAD_CONST 0 (None)
 6 RETURN_VALUE

 5 >> 8 RERAISE
 10 LOAD_CONST 0 (None)
 12 RETURN_VALUE

[markshannon]

I believe the current behavior is correct.

def foo() -> int:
 with open('/dev/null') as devnull:
 return 'a'

The sequence of events is:

• Function start at line 1
• Line 2
• Evaluate open and enter the with block
• Line 3
• Evaluate 'a' and the return statement
• Line 2
• Exit the with block

The jump back to line 2 may seem surprising, but it is necessary.
Because the call to __exit__ when exiting the with statement is is observable and may raise an exception, it needs to have a line number.

[markshannon]

If you change "a" to len("a"), note that len("a") is on line 3.

def foo() -> int:
 with open('/dev/null') as devnull:
 return len("a")

 1 0 RESUME 0

 2 2 LOAD_GLOBAL 1 (NULL + open)
 14 LOAD_CONST 1 ('/dev/null')
 16 CALL 1
 26 BEFORE_WITH
 28 STORE_FAST 0 (devnull)

 3 30 LOAD_GLOBAL 3 (NULL + len)
 42 LOAD_CONST 2 ('a')
 44 CALL 1

 2 54 SWAP 2
 56 LOAD_CONST 0 (None)
 58 LOAD_CONST 0 (None)
 60 LOAD_CONST 0 (None)
 62 CALL 2
 72 POP_TOP
 74 RETURN_VALUE
 >> 76 PUSH_EXC_INFO
 78 WITH_EXCEPT_START
 80 POP_JUMP_FORWARD_IF_TRUE 1 (to 84)
 82 RERAISE 2
 >> 84 POP_TOP
 86 POP_EXCEPT
 88 POP_TOP
 90 POP_TOP
 92 LOAD_CONST 0 (None)
 94 RETURN_VALUE
 >> 96 COPY 3
 98 POP_EXCEPT
 100 RERAISE 1
ExceptionTable:
 28 to 52 -> 76 [1] lasti
 76 to 84 -> 96 [3] lasti

[markshannon]

On more thing.
In Python 3.9, if __exit__ raises, it reports the wrong line number.

def foo() -> int:
 with raises_on_exit() as mgr:
 return 'a'
 pass

Will report the exception on line 4, which is clearly wrong.

It is 3.9 that is broken, not 3.10 or 3.11, IMO.