Skip to content

gh-131423: Update OpenSSL data to 3.4.1 on Linux#131618

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
picnixz merged 15 commits into from
Apr 25, 2025
Merged

gh-131423: Update OpenSSL data to 3.4.1 on Linux #131618

picnixz merged 15 commits into from
Apr 25, 2025

Conversation

@picnixz
Copy link
Member

@picnixzpicnixz commented Mar 23, 2025•
edited
Loading

I've also updated the make_ssl_data.py script that @encukou has recently updated as well. I completed with instructions that I thought usefull for future maintainers.

📚 Documentation preview 📚: https://cpython-previews--131618.org.readthedocs.build/

@picnixz
update OpenSSL CI build configuration
85ec7b2
@picnixz
update OpenSSL mnemonics
40ad7ce
Since mnemonics from 3.4.1 are different (renumbered) from 3.4.0. To ease future updates, we assume the following: `_ssl_data_<MAJOR><PATCH>.h` contains the latest OpenSSL data. If the previous `_ssl_data_<MAJOR><PATCH>.h` file is incompatible with the newest one (e.g., because some mnemonics were renamed or removed), the old one is renamed to `_ssl_data_<MAJOR><MINOR><PATCH>.h` where <PATCH> is the patch number it was based upon. In this commit, OpenSSL 3.4.1 mnemonics are not compatible with OpenSSL 3.4.0 mnemonics as they were renumbered. Therefore, `_ssl_data_34.h` is renamed to `_ssl_data_340.h` and `_ssl_data_34x.h` now contains OpenSSL 3.4.1 mnemonics. We also refined the mnemonics that are selected, discarding those that are mnemonics-like but should not be used as such (e.g., ERR_LIB_MASK and ERR_LIB_OFFSET for OpenSSL 1.1.1).
@picnixz
update macOS and Windows OpenSSL configurations
b35b7fa
@picnixz
Merge remote-tracking branch 'upstream/main' into ci/update/ssl-versi…
578bb0e
…ons-131423
@picnixz
update SBOM
296a98c
@picnixz
update SBOM
326d947
@picnixz
update configure requirements
cf7a65d
@bedevere-appbedevere-appbot mentioned this pull request Mar 23, 2025
Closed
4 tasks
@picnixzpicnixzforce-pushed the ci/update/ssl-versions-131423 branch from 05ee142 to 5bbc702CompareMarch 23, 2025 10:53
@picnixzpicnixz marked this pull request as ready for review March 24, 2025 14:56
encukou
encukou reviewed Mar 24, 2025
View reviewed changes
Tools/ssl/make_ssl_data.py
Comment on lines +146 to +147
# FEAT(picnixz): in the future, we may want to also check
# the consistency of the OpenSSL files with an external tool.
Copy link
Member

@encukouencukouMar 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you track that in an issue instead?

Copy link
MemberAuthor

@picnixzpicnixzApr 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tracked in #132745.

picnixz
picnixz commented Mar 24, 2025
View reviewed changes
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
picnixz
picnixz commented Mar 24, 2025
View reviewed changes
@picnixzpicnixz marked this pull request as draft March 24, 2025 17:49
@picnixz
Copy link
MemberAuthor

picnixz commented Mar 24, 2025

arf, I'm not on my Linux so I can't regen :< I'm leaving tomorrow morning so I'm not really sure I'll be able to commit before leaving, but otherwise, just take over the PR and regen the data!

picnixz
picnixz commented Mar 24, 2025
View reviewed changes
ned-deily
ned-deily reviewed Mar 26, 2025
View reviewed changes
Copy link
Member

@ned-deilyned-deily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The macOS build-installer.py changes LGTM, thanks!

@picnixzpicnixz requested a review from encukouMarch 28, 2025 22:50
@picnixzpicnixz requested a review from gpshead as a code ownerApril 5, 2025 10:52
@picnixzpicnixz changed the title gh-131423: Update OpenSSL to 3.0.16 (macOS, Windows) and 3.4.1 (Linux)gh-131423: Update OpenSSL to 3.0.16 (macOS) and 3.4.1 (Linux)Apr 5, 2025
@picnixzpicnixz added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 5, 2025
@bedevere-bot
Copy link

bedevere-bot commented Apr 5, 2025

🤖 New build scheduled with the buildbot fleet by @picnixz for commit 905f1a5 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F131618%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 5, 2025
@picnixz
Copy link
MemberAuthor

picnixz commented Apr 5, 2025

To avoid surprises, I'm running the build bots. If they pass, I'll merge this one so that we can close the other issue. I think the Windows-related failures were recently solved as well

@picnixz
Copy link
MemberAuthor

picnixz commented Apr 5, 2025•
edited by hugovk
Loading

The iOS failure is known (PR #132050)

@picnixz
Copy link
MemberAuthor

picnixz commented Apr 6, 2025

I want to think about something. Mnenmonics were updated in 3.4.1 compared to 3.4.0, but that's only because I knew that they were changed. However, we're actually having a

#if (OPENSSL_VERSION_NUMBER >= 0x30100000L) #include"_ssl_data_34.h"

So I think I'll need a way to check first that when OpenSSL mnemonics changed so that we regenerate the correct files per version.

@ned-deily
Copy link
Member

ned-deily commented Apr 6, 2025

@picnixz, with the 3.14.a7, 3.13.3, and 3.12.10 releases approaching in two days, I plan to update the macOS installers for those releases to use 3.0.16. If you don't expect to be able to merge this PR before then, I can pull out the build-installer.py change into a separate PR since it has no relation to any of the other changes in this PR. (And that's why I prefer to keep changes like this separate.)

@picnixz
Copy link
MemberAuthor

picnixz commented Apr 6, 2025

If you don't expect to be able to merge this PR before then

To be on the safe side, please do so. I don't want to block the macOS side with my interrogations. Hopefully I'll be able to merge this before the release.

ned-deily
ned-deily reviewed Apr 7, 2025
View reviewed changes
ned-deily
ned-deily reviewed Apr 7, 2025
View reviewed changes
@picnixz
Copy link
MemberAuthor

picnixz commented Apr 7, 2025•
edited
Loading

Note: 3.4.1 includes CVE patches but since we're still in alpha, we can say that those security patches are not really necessary now (in addition, they affect components that are not directly exposed by Python IIRC). So, I'll postpone this until after the release (I don't want to have surprises where a mnemonic change would be actually annoying for a user)

@picnixzpicnixz changed the title gh-131423: Update OpenSSL to 3.0.16 (macOS) and 3.4.1 (Linux)gh-131423: Update OpenSSL to 3.4.1 on LinuxApr 7, 2025
@picnixzpicnixz changed the title gh-131423: Update OpenSSL to 3.4.1 on Linuxgh-131423: Update OpenSSL data to 3.4.1 on LinuxApr 8, 2025
@picnixz
Copy link
MemberAuthor

picnixz commented Apr 25, 2025

I'll merge this one and work on #132745.

@picnixzpicnixz merged commit 6a9bfee into python:mainApr 25, 2025
45 checks passed
@picnixzpicnixz deleted the ci/update/ssl-versions-131423 branch April 25, 2025 08:27
@picnixz
Copy link
MemberAuthor

picnixz commented Apr 25, 2025

Rationale for not backporting: #131423 (comment).

@zwarezware mentioned this pull request Aug 14, 2025
Merged
@encukouencukou mentioned this pull request Aug 15, 2025
Merged
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zoobazoobazooba left review comments

@ned-deilyned-deilyned-deily left review comments

@gpsheadgpsheadgpshead approved these changes

@erlend-aaslanderlend-aaslandAwaiting requested review from erlend-aaslanderlend-aasland is a code owner

@corona10corona10Awaiting requested review from corona10corona10 is a code owner

@ezio-melottiezio-melottiAwaiting requested review from ezio-melottiezio-melotti is a code owner

@hugovkhugovkAwaiting requested review from hugovkhugovk is a code owner

@AA-TurnerAA-TurnerAwaiting requested review from AA-TurnerAA-Turner is a code owner

@encukouencukouAwaiting requested review from encukou

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

6 participants

@picnixz@bedevere-bot@ned-deily@gpshead@encukou@zooba