gh-103578: Fix pdb reading code with non-utf8 encoding

[gaogaotiantian]

pdb should use io.open_code to open code to avoid encoding issue.

• Issue: python -m pdb <utf-8 encoding file.py> return UnicodeDecodeError on Windows 8.1/10 (code page 936) #103578

[hauntsaninja]

Thanks for this PR!

We should probably mention in the news entry that because we use io.open_code, we'll now trigger auditing events hooks

[gaogaotiantian]

Thanks for this PR!

We should probably mention in the news entry that because we use io.open_code, we'll now trigger auditing events

io.open() also raise auditing events. According to the docs,

However, assuming that path is a str and an absolute path, open_code(path) should always behave the same as open(path, 'rb')

io.open_code() should behave the same as io.open(path, 'rb'), so is there an auditing event difference?

[hauntsaninja]

Ah, I meant the PyFile_SetOpenCodeHook() hook from PEP 578. I'm not sure what the best phrase is, but it was described as auditing event in #17127. Maybe "auditing hook" as per the PEP?
(that PR got regressed in #26992, this PR fixes :-) )

[gaogaotiantian]

So it seems like changing it to io.open_code() makes it possible to call into the hook set by PyFile_SetOpenCodeHook()? Is that something expected in the news entry?

[hauntsaninja]

Yup! I think can't hurt to mention it in the news entry, since it is a (good) change in behaviour and one that seems potentially noteworthy.

[gaogaotiantian]

Yup! I think can't hurt to mention it in the news entry, since it is a (good) change in behaviour and one that seems potentially noteworthy.

I updated the news entry. Let me know if you think that requires extra polish!

[hauntsaninja]

Looks great, thanks!

[miss-islington]

Thanks @gaogaotiantian for the PR, and @hauntsaninja for merging it 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

[bedevere-bot]

GH-103867 is a backport of this pull request to the 3.11 branch.

[miss-islington]

Thanks @gaogaotiantian for the PR, and @hauntsaninja for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

[miss-islington]

Thanks @gaogaotiantian for the PR, and @hauntsaninja for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9.
🐍🍒⛏🤖

[miss-islington]

Thanks @gaogaotiantian for the PR, and @hauntsaninja for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

[miss-islington]

Sorry @gaogaotiantian and @hauntsaninja, I had trouble checking out the 3.8 backport branch.
Please retry by removing and re-adding the "needs backport to 3.8" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 31acfd78a0810f84898d36a8289e407d3754b823 3.8

[miss-islington]

Sorry, @gaogaotiantian and @hauntsaninja, I could not cleanly backport this to 3.9 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 31acfd78a0810f84898d36a8289e407d3754b823 3.9

[miss-islington]

Sorry @gaogaotiantian and @hauntsaninja, I had trouble checking out the 3.10 backport branch.
Please retry by removing and re-adding the "needs backport to 3.10" label.
Alternatively, you can backport using cherry_picker on the command line.
cherry_picker 31acfd78a0810f84898d36a8289e407d3754b823 3.10

[miss-islington]

Thanks @gaogaotiantian for the PR, and @hauntsaninja for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

[miss-islington]

Sorry, @gaogaotiantian and @hauntsaninja, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 31acfd78a0810f84898d36a8289e407d3754b823 3.10

[zooba]

No backport needed - this was fine in 3.8-3.10 and was regressed in 3.11.