gh-125008: Fix tokenize.untokenize roundtrip for \n{{

[tomasr8]

I'm not sure if this is the best way to fix this issue so I'm open to alternatives :)

• Issue: Tokenize does not roundtrip{{after \n #125008

[wyattscarpenter]

Might as well add test cases for \ alone, while you're at it. Hope this is helpful, if you do decide to do that:

self.check_roundtrip(r"rf'\t{{foo}}'") self.check_roundtrip(r"f'\\t{{foo}}'") self.check_roundtrip(r"rf'\\\t{{foo}}'") self.check_roundtrip(r"f'\\\\t{{foo}}'")

(some of them are raw to avoid SyntaxWarning: invalid escape sequence '\{'.

[pablogsal]

The changes LGTM. Thanks a lot for the catch and the fix @tomasr8!

I will wait if you want to address @wyattscarpenter suggestion and we can land afterwards

[pablogsal]

This will need backport to 3.12 and 3.13

[tomasr8]

Thanks for the suggestion @wyattscarpenter ! I added more tests with raw strings :)

[wyattscarpenter]

Sorry, I bungled the code example I was trying to give you. I actually meant

self.check_roundtrip(r"rf'\{{foo}}'") self.check_roundtrip(r"f'\\{{foo}}'") self.check_roundtrip(r"rf'\\\{{foo}}'") self.check_roundtrip(r"f'\\\\{{foo}}'")

\\ currently suffers from the same defect of the disappearing{, like \t, but it's sort of a special case so it might be worth testing for explicitly. Or not; it's not that special of a case so maybe it doesn't matter.

[tomasr8]

Added more tests :) f-strings are tricky so I think it's better to have more tests anyway

[miss-islington-app]

Thanks @tomasr8 for the PR, and @pablogsal for merging it 🌮🎉.. I'm working now to backport this PR to: 3.12, 3.13.
🐍🍒⛏🤖

[pablogsal]

Landing! Great job

[bedevere-app]

GH-125020 is a backport of this pull request to the 3.13 branch.

[bedevere-app]

GH-125021 is a backport of this pull request to the 3.12 branch.