Empty unclosed f-string does not directly raise a SyntaxError in the REPL

[lysnikolaou]

Bug report

This is another bug introduced in 1ef61cf. When typing an empty single-quoted unclosed f-string in the REPL, another line is added and input is expected, before a SyntaxError is raised.

Before 1ef61cf:

❯ ./python.exe 
Python 3.12.0a7+ (tags/v3.12.0a7-153-ga6b07b5a34:a6b07b5a34, Apr 23 2023, 13:18:40) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> f'
 File "<stdin>", line 1
 f'
 ^
SyntaxError: unterminated string literal (detected at line 1)

Current main:

❯ ./python.exe 
Python 3.12.0a7+ (heads/main:05b3ce7339, Apr 23 2023, 13:26:24) [Clang 14.0.3 (clang-1403.0.22.14.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> f"
... 
 File "<stdin>", line 1
 f"
 ^
SyntaxError: unterminated f-string literal (detected at line 2)
>>> f"
... asdnc
 File "<stdin>", line 1
 f"
 ^
SyntaxError: unterminated f-string literal (detected at line 2)
>>> f"
... asdasd"
 File "<stdin>", line 1
 f"
 ^
SyntaxError: unterminated f-string literal (detected at line 2)

Linked PRs

• GH-103727: Avoid advancing tokenizer too far in f-string mode #103775

[terryjreedy]

In IDLE, which uses code.interactive_interpreter, which ultimately uses codeop._maybe_compile, I get 'SyntaxError: incomplete input' in installed 3.12.0a7 and the new message at line 1 in fresh repository 3.12.0a7+.

[lysnikolaou]

codeop._maybe_compile uses compile, which doesn't encounter the problem mentioned here.

It's got to do with the interactive mode, where we advance the tokenizer far enough (to backup again afterwards) that a newline character alone is not enough for the syntax error to be emitted directly.

[lysnikolaou]

Resolved in #103775.