gh-132661: Disallow `Template`/`str` concatenation after PEP 750 spec update#135996

davepeck · 2025-06-26T18:23:53Z

Following the steering council decision and corresponding update to PEP750, we are removing support for both implicit and explicit Template/str concatenation.

Issue: Implement PEP 750 – Template Strings #132661

sobolevn

Thank you!

Parser/action_helpers.c

Objects/unicodeobject.c

Lib/test/test_tstring.py

lysnikolaou · 2025-06-30T15:35:06Z

Happy to do this with your input, or happy to hand it off to you!

Happy to help out / review this PR if you wanna take a stab at it. Feel free to reach out if you have any specific questions.

Today was, alas, not that day.

davepeck · 2025-07-08T23:28:49Z

@lysnikolaou Think this is ready. The only question for me is whether we want to go further with _PyPegen_concatenate_strings().

sobolevn

Thank you! Several small nitpicks :)

Lib/test/test_tstring.py

Misc/NEWS.d/next/Core_and_Builtins/2025-07-08-23-22-08.gh-issue-132661.34ftJl.rst

Objects/templateobject.c

lysnikolaou

Thanks @davepeck for working on this! Looks great in general. Left a few unimportant comments and one more significant one about how to implement this in the parser. Also, we'll need to change ast_unparse.c around https://github.com/python/cpython/blob/main/Python/ast_unparse.c#L715.

Lib/test/test_tstring.py

Objects/templateobject.c

Parser/action_helpers.c

Lib/test/test_tstring.py

efimov-mikhail · 2025-07-16T17:49:50Z

LGTM

lysnikolaou · 2025-07-17T11:21:53Z

Looks great and aaalmost there. I have one last thing I missed during my previous reviews.

If I remember correctly, https://github.com/python/cpython/blob/main/Python/codegen.c#L4084-L4093 can be removed as well, because it's only there to handle FormattedValues, which cannot happen anymore.

After that, it's gonna be good to go, promise!

davepeck · 2025-07-17T15:59:35Z

If I remember correctly, https://github.com/python/cpython/blob/main/Python/codegen.c#L4084-L4093 can be removed as well, because it's only there to handle FormattedValues, which cannot happen anymore.

@lysnikolaou Ah, yes -- thank you. Removed now; tests continue to pass.

(If we prefer, I can replace VISIT() in the else block with a direct ADDOP_LOAD_CONST(). I'd probably add an assert(value->kind == Constant_kind) there, too.)

lysnikolaou

Looks good to me! Thanks @davepeck! Great work.

miss-islington-app · 2025-07-21T06:44:29Z

Thanks @davepeck for the PR, and @lysnikolaou for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

miss-islington-app · 2025-07-21T06:44:32Z

Sorry, @davepeck and @lysnikolaou, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker c5e77af131aa0c8832a9ee50c4410731254e4209 3.14

lysnikolaou · 2025-07-21T06:45:10Z

I can handle tha backport.

…0 spec update (python#135996) Co-authored-by: sobolevn <mail@sobolevn.me> Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> (cherry picked from commit c5e77af)

…50 spec update (#135996) (#136901) Co-authored-by: Dave Peck <davepeck@gmail.com> Co-authored-by: sobolevn <mail@sobolevn.me>

…19485) As of [this cpython PR](python/cpython#135996), it is not allowed to concatenate t-strings with non-t-strings, implicitly or explicitly. Expressions such as `"foo" t"{bar}"` are now syntax errors. This PR updates some AST nodes and parsing to reflect this change. The structural change is that `TStringPart` is no longer needed, since, as in the case of `BytesStringLiteral`, the only possibilities are that we have a single `TString` or a vector of such (representing an implicit concatenation of t-strings). This removes a level of nesting from many AST expressions (which is what all the snapshot changes reflect), and simplifies some logic in the implementation of visitors, for example. The other change of note is in the parser. When we meet an implicit concatenation of string-like literals, we now count the number of t-string literals. If these do not exhaust the total number of implicitly concatenated pieces, then we emit a syntax error. To recover from this syntax error, we encode any t-string pieces as _invalid_ string literals (which means we flag them as invalid, record their range, and record the value as `""`). Note that if at least one of the pieces is an f-string we prefer to parse the entire string as an f-string; otherwise we parse it as a string. This logic is exactly the same as how we currently treat `BytesStringLiteral` parsing and error recovery - and carries with it the same pros and cons. Finally, note that I have not implemented any changes in the implementation of the formatter. As far as I can tell, none are needed. I did change a few of the fixtures so that we are always concatenating t-strings with t-strings.

…0 spec update (python#135996) Co-authored-by: sobolevn <mail@sobolevn.me> Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>

serhiy-storchaka · 2025-08-14T18:11:42Z

Reminder about backporting. @davepeck @lysnikolaou

encukou · 2025-08-15T10:12:11Z

GH-136901 is a backport of this pull request to the 3.14 branch.

…0 spec update (python#135996) Co-authored-by: sobolevn <mail@sobolevn.me> Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>

… PEP 750 spec update (python#135996) (python#136901) Co-authored-by: Dave Peck <davepeck@gmail.com> Co-authored-by: sobolevn <mail@sobolevn.me>

davepeck added 2 commits June 26, 2025 17:46

Disallow explicit concat between Template and str
5d5e187

First pass at removing implicit Template/str concat
1cc778f

bedevere-appbot mentioned this pull request Jun 26, 2025
Implement PEP 750 – Template Strings #132661
Closed

sobolevn reviewed Jun 26, 2025
View reviewed changes

Parser/action_helpers.c Outdated Show resolvedHide resolved
Objects/unicodeobject.cShow resolvedHide resolved
Lib/test/test_tstring.py Outdated Show resolvedHide resolved

Merge branch 'main' into pep750-concat-update
7ea498c

dylwil3 mentioned this pull request Jul 4, 2025
Add t-string fixtures for rules that do not need to be modified astral-sh/ruff#19146
Merged

davepeck added 8 commits July 8, 2025 12:50

Merge branch 'main' into pep750-concat-update
d08c8b8

Remove extraneous newline
c662041

Add expected exception messages to tests
b17b7c9

Remove _ast_unparse t/f implicit concat helper code
a0c1bb6

Merge branch 'main' into pep750-concat-update
23988e8

Add a parallel comment
d7eadec

Add news blurb for PR.
cb1ad63

One day, I may develop RST muscle memory.
741eaf2
Today was, alas, not that day.

davepeck marked this pull request as ready for review July 8, 2025 23:27

davepeck requested review from Eclips4, JelleZijlstra, isidentical, lysnikolaou, pablogsal and tomasr8 as code owners July 8, 2025 23:27

bedevere-appbot added the awaiting review label Jul 8, 2025

Merge branch 'main' into pep750-concat-update
8aeb512

JelleZijlstra approved these changes Jul 9, 2025
View reviewed changes

bedevere-appbot added awaiting merge and removed awaiting review labels Jul 9, 2025

sobolevn reviewed Jul 9, 2025
View reviewed changes

lysnikolaou requested changes Jul 9, 2025
View reviewed changes

Lib/test/test_tstring.py Outdated Show resolvedHide resolved
Objects/templateobject.c Outdated Show resolvedHide resolved
Parser/action_helpers.c Outdated Show resolvedHide resolved

efimov-mikhail reviewed Jul 16, 2025
View reviewed changes

Lib/test/test_tstring.py Outdated Show resolvedHide resolved

efimov-mikhail reviewed Jul 16, 2025
View reviewed changes

Lib/test/test_tstring.py Outdated Show resolvedHide resolved

Fix exception assert message checks
6ea3af3

Remove no-longer used path in codegen.c
5037f89

davepeck requested review from iritkatriel and markshannon as code owners July 17, 2025 15:59

lysnikolaou approved these changes Jul 21, 2025
View reviewed changes

bedevere-appbot added awaiting merge and removed awaiting change review labels Jul 21, 2025

lysnikolaou added the needs backport to 3.14 bugs and security fixes label Jul 21, 2025

lysnikolaou merged commit c5e77af into python:mainJul 21, 2025
46 of 47 checks passed

lysnikolaou deleted the pep750-concat-update branch July 21, 2025 06:44

bedevere-appbot removed the awaiting merge label Jul 21, 2025

miss-islington-appbot assigned lysnikolaou Jul 21, 2025

lysnikolaou added a commit that referenced this pull request Jul 21, 2025
[3.14]gh-132661: DisallowTemplate/strconcatenation after PEP 7…
af8d1f5
…50 spec update (#135996) (#136901) Co-authored-by: Dave Peck <davepeck@gmail.com> Co-authored-by: sobolevn <mail@sobolevn.me>

dylwil3 mentioned this pull request Jul 25, 2025
Disallow implicit concatenation of t-strings and other string types astral-sh/ruff#19485
Merged

encukou removed the needs backport to 3.14 bugs and security fixes label Aug 15, 2025

Uh oh!

gh-132661: Disallow Template/str concatenation after PEP 750 spec update#135996

gh-132661: Disallow Template/str concatenation after PEP 750 spec update #135996

Uh oh!

Conversation

davepeck commented Jun 26, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lysnikolaou commented Jun 30, 2025

Uh oh!

davepeck commented Jul 8, 2025

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lysnikolaou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

efimov-mikhail commented Jul 16, 2025

Uh oh!

lysnikolaou commented Jul 17, 2025

Uh oh!

davepeck commented Jul 17, 2025• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lysnikolaou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-appbot commented Jul 21, 2025

Uh oh!

miss-islington-appbot commented Jul 21, 2025

Uh oh!

lysnikolaou commented Jul 21, 2025

Uh oh!

serhiy-storchaka commented Aug 14, 2025

Uh oh!

encukou commented Aug 15, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

8 participants

gh-132661: Disallow `Template`/`str` concatenation after PEP 750 spec update#135996

gh-132661: Disallow `Template`/`str` concatenation after PEP 750 spec update #135996

davepeck commented Jun 26, 2025•
edited
Loading

davepeck commented Jul 17, 2025•
edited
Loading