gh-127257: ssl: Raise OSError for ERR_LIB_SYS#127361

encukou · 2024-11-28T13:19:38Z

One way OpenSSL can report a failed syscall is SSL_ERROR_SYSCALL. We have that one covered.

But, another way is ERR_LIB_SYS. Apparently, it's being used more now, and causing issues on OpenSSL 3.4.0 on Arch buildbots.

From the ERR_raise manpage:

ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`.

This PR only handles ERR_LIB_SYS for the high-lever error types
SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where
OpenSSL indicates it has some more information about the issue.

I believe it's correct to raise OSError rather that SSLError
in these cases. That makes this a backwards-incompatible change
(but one we still might want to backport as a bugfix).

The error on Arch with OpenSSL 3.4.0 is broken pipe (EPIPE) when a client reads after the server shuts down. On my Arch, raising OSError makes test_poplip pass.

Issue: [tests] test_poplib fails with "env changed" on Arch Linux with OpenSSL 3.4: [SYS] unknown error (_ssl.c:2634) #127257

From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`.

encukou · 2024-11-28T13:35:47Z

!buildbot Arch.Linux

bedevere-bot · 2024-11-28T13:35:49Z

🤖 New build scheduled with the buildbot fleet by @encukou for commit 0098897 🤖

The command will test the builders whose names match following regular expression: Arch.Linux

The builders matched are:

AMD64 Arch Linux Perf PR
AMD64 Arch Linux Asan PR
AMD64 Arch Linux VintageParser PR
AMD64 Arch Linux Valgrind PR
AMD64 Arch Linux Usan PR
AMD64 Arch Linux Usan Function PR
AMD64 Arch Linux Asan Debug PR
AMD64 Arch Linux TraceRefs PR

petermarko · 2024-11-28T20:07:53Z

This fixes #125936 for me.
Thanks!

python/cpython#127331 python/cpython#127361 (From OE-Core rev: e271e9cbf896f1fb97d56c426e4217a6d2105ea4) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

python/cpython#127331 python/cpython#127361 Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

python/cpython#127331 python/cpython#127361 (From OE-Core rev: e5f3a1793e34fb4cd1e53ca60b67f9a9f084b7a6) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

python/cpython#127331 python/cpython#127361 Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

encukou · 2024-12-04T09:05:23Z

ssl experts (@jackjansen, @tiran, @dstufft, @alex): Do you want to check the change?
I think this is the correct place, and the correct error to raise, but I'd appreciate someone with domain knowledge.

encukou · 2024-12-09T12:31:40Z

If there are no objections, I'll merge tomorrow to unblock the buildbots.

vstinner

The change looks good to me. I don't know how to test it in a reliable way. Let's see how it goes with EPIPE and OpenSSL 3.4.

miss-islington-app · 2024-12-11T08:18:57Z

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖

From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) Co-authored-by: Petr Viktorin <[email protected]>

bedevere-app · 2024-12-11T08:19:13Z

GH-127812 is a backport of this pull request to the 3.13 branch.

…127812) gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) Co-authored-by: Petr Viktorin <[email protected]>

miss-islington-app · 2024-12-12T11:45:15Z

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington-app · 2024-12-13T11:03:26Z

Thanks @encukou for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) Co-authored-by: Petr Viktorin <[email protected]>

bedevere-app · 2024-12-13T11:03:39Z

GH-127905 is a backport of this pull request to the 3.12 branch.

…127905) gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) Co-authored-by: Petr Viktorin <[email protected]>

python/cpython#127331 python/cpython#127361 (From OE-Core rev: e5f3a1793e34fb4cd1e53ca60b67f9a9f084b7a6) Signed-off-by: Peter Marko <[email protected]> Signed-off-by: Richard Purdie <[email protected]>

From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue.

OpenSSL 3.4.0 returns `ERR_LIB_SYS` in some more situations than it used to. In the case exercised by `SingleTLSLayerTestCase.test_close_after_handshake`, python/cpython#127361 (also backported to the 3.12 and 3.13 branches) turns this into `BrokenPipeError`. It seems reasonable to treat this in the same way as `ConnectionAbortedError` and `ConnectionResetError`.

stratakis · 2025-04-01T10:13:32Z

We started seeing this on 3.11 and 3.10 in Fedora Rawhide which moved to OpenSSL 3.5 beta (due to unrelated to this issue ssl test failures). Would this be considered a security fix to backport for the security releases as more and more distros start moving to the latest OpenSSL?

…127361) (pythonGH-127905) pythongh-127257: ssl: Raise OSError for ERR_LIB_SYS (pythonGH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) Co-authored-by: Petr Viktorin <[email protected]> (cherry picked from commit 7f707fa)

vstinner · 2025-04-01T10:45:28Z

We started seeing this on 3.11 and 3.10 in Fedora Rawhide which moved to OpenSSL 3.5 beta (due to unrelated to this issue ssl test failures). Would this be considered a security fix to backport for the security releases as more and more distros start moving to the latest OpenSSL?

IMO we can make an exception for this change and backport it to 3.11 and 3.10. I prepared backports:

3.11: [3.11] gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) (GH-127905) #131970
3.10: [3.10] gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) (GH-127905) #131971

…127905) (GH-131970) gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) (cherry picked from commit 7f707fa) Co-authored-by: Petr Viktorin <[email protected]>

…127905) (GH-131971) gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) (cherry picked from commit 7f707fa) Co-authored-by: Petr Viktorin <[email protected]>

…pythonGH-127905) (pythonGH-131971) pythongh-127257: ssl: Raise OSError for ERR_LIB_SYS (pythonGH-127361) From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`. This PR only handles ERR_LIB_SYS for the high-lever error types SSL_ERROR_SYSCALL and SSL_ERROR_SSL, i.e., not the ones where OpenSSL indicates it has some more information about the issue. (cherry picked from commit f4b31ed) (cherry picked from commit 7f707fa) Co-authored-by: Petr Viktorin <[email protected]>

The patch resolves the flakiness of test_ftplib Backported from upstream 3.10+: python#127361 Co-authored-by: Petr Viktorin <[email protected]>

ssl: Raise OSError for ERR_LIB_SYS
11e0523
From the ERR_raise manpage: ERR_LIB_SYS This "library code" indicates that a system error is being reported. In this case, the reason code given to `ERR_raise()` and `ERR_raise_data()` *must* be `errno(3)`.

encukou requested a review from gpshead November 28, 2024 13:19

bedevere-appbot mentioned this pull request Nov 28, 2024
[tests] test_poplib fails with "env changed" on Arch Linux with OpenSSL 3.4: [SYS] unknown error (_ssl.c:2634) #127257
Closed

bedevere-appbot added the awaiting core review label Nov 28, 2024

Add blurb
0098897

vstinner approved these changes Dec 9, 2024
View reviewed changes

bedevere-appbot added awaiting merge and removed awaiting core review labels Dec 9, 2024

encukou merged commit f4b31ed into python:mainDec 10, 2024
4 checks passed

encukou deleted the ssl-sys-error branch December 10, 2024 10:56

bedevere-appbot removed the awaiting merge label Dec 10, 2024

encukou added the needs backport to 3.13 bugs and security fixes label Dec 11, 2024

bedevere-appbot removed the needs backport to 3.13 bugs and security fixes label Dec 11, 2024

encukou added the needs backport to 3.12 only security fixes label Dec 12, 2024

petermarko mentioned this pull request Dec 12, 2024
openssl-3.4.0 compatibility #125936
Closed

encukou removed the needs backport to 3.12 only security fixes label Dec 13, 2024

encukou added the needs backport to 3.12 only security fixes label Dec 13, 2024

bedevere-appbot removed the needs backport to 3.12 only security fixes label Dec 13, 2024

cjwatson mentioned this pull request Jan 9, 2025
SingleTLSLayerTestCase: Catch BrokenPipeError urllib3/urllib3#3547
Merged

lblasc mentioned this pull request Feb 17, 2025
python310/python311: fix failing tests with openssl >= 3.4 NixOS/nixpkgs#382794
Merged
13 tasks

bdarnell mentioned this pull request Mar 7, 2025
BaseIOStream.read_bytes may raise incorrect exception type with OpenSSL 3.4.0 tornadoweb/tornado#3443
Closed

vstinner mentioned this pull request Apr 1, 2025
[3.11] gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) (GH-127905) #131970
Merged

vstinner mentioned this pull request Apr 1, 2025
[3.10] gh-127257: ssl: Raise OSError for ERR_LIB_SYS (GH-127361) (GH-127905) #131971
Merged

stratakis mentioned this pull request Apr 14, 2025
test_ssl: ConnectionHandler shuts down ThreadedEchoServer #126500
Closed

stratakis pushed a commit to stratakis/cpython that referenced this pull request Apr 16, 2025
00457: ssl: Raise OSError for ERR_LIB_SYS
04ae7c2
The patch resolves the flakiness of test_ftplib Backported from upstream 3.10+: python#127361 Co-authored-by: Petr Viktorin <[email protected]>

stratakis mentioned this pull request Apr 16, 2025
00457: ssl: Raise OSError for ERR_LIB_SYS fedora-python/cpython#115
Merged

pmgdeb mentioned this pull request Apr 22, 2025
Deal with BrokenPipeError since openssl 3.4.x. gevent/gevent#2103
Closed

hrnciar pushed a commit to fedora-python/cpython that referenced this pull request Apr 23, 2025
00457: ssl: Raise OSError for ERR_LIB_SYS
6031114
The patch resolves the flakiness of test_ftplib Backported from upstream 3.10+: python#127361 Co-authored-by: Petr Viktorin <[email protected]>

hroncok pushed a commit to fedora-python/cpython that referenced this pull request Jul 4, 2025
00457: ssl: Raise OSError for ERR_LIB_SYS
896e709
The patch resolves the flakiness of test_ftplib Backported from upstream 3.10+: python#127361 Co-authored-by: Petr Viktorin <[email protected]>

frenzymadness pushed a commit to fedora-python/cpython that referenced this pull request Aug 12, 2025
00457: ssl: Raise OSError for ERR_LIB_SYS
5f5badb
The patch resolves the flakiness of test_ftplib Backported from upstream 3.10+: python#127361 Co-authored-by: Petr Viktorin <[email protected]>

Uh oh!

gh-127257: ssl: Raise OSError for ERR_LIB_SYS#127361

gh-127257: ssl: Raise OSError for ERR_LIB_SYS #127361

Uh oh!

Conversation

encukou commented Nov 28, 2024• edited by bedevere-app botLoading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

encukou commented Nov 28, 2024

Uh oh!

bedevere-bot commented Nov 28, 2024

Uh oh!

petermarko commented Nov 28, 2024

Uh oh!

encukou commented Dec 4, 2024

Uh oh!

encukou commented Dec 9, 2024

Uh oh!

vstinner left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

miss-islington-appbot commented Dec 11, 2024

Uh oh!

bedevere-appbot commented Dec 11, 2024

Uh oh!

miss-islington-appbot commented Dec 12, 2024

Uh oh!

miss-islington-appbot commented Dec 13, 2024

Uh oh!

bedevere-appbot commented Dec 13, 2024

Uh oh!

stratakis commented Apr 1, 2025

Uh oh!

vstinner commented Apr 1, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

encukou commented Nov 28, 2024•
edited by bedevere-app bot
Loading