gh-98739: Update libexpat from 2.4.9 to 2.5.0

[scdub]

Based this PR on the recent earlier upgrade at #97006 by @corona10

• Issue: Upgrade bundled expat to 2.5.0 #98739

[bedevere-bot]

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

[scdub]

@hartwork I'm not sure if there are other changes that you think would be necessary for Python to upgrade to your latest release, but here's a first attempt at to get Python synced up. I believe there is a constellation of security fixes which Python will be requiring in the near term (SQLite, OpenSSL, ...) and this could potentially tag along for one of those releases.

[hartwork]

@scdub I don't really understand your message. Can you maybe rephrase your question for me to better understand?

[hartwork]

Verified using this Dockerfile:

# Copyright (c) 2022 Sebastian Pipping <sebastian@pipping.org># Licensed under the Apache License version 2.0FROM alpine RUN apk add --update \ diffutils \ git \ sed \ && \ git clone --depth 1 https://github.com/python/cpython cpython-main \ && \ ( cd cpython-main && git rev-parse HEAD ) \ && \ git clone --depth 1 --branch expat-2.5.0 https://github.com/scdub/cpython scdub-expat-2-5-0 \ && \ ( cd scdub-expat-2-5-0 && git rev-parse HEAD ) \ && \ git config --global advice.detachedHead false \ && \ git clone --depth 1 --branch R_2_4_9 https://github.com/libexpat/libexpat libexpat_2_4_9 \ && \ git clone --depth 1 --branch R_2_5_0 https://github.com/libexpat/libexpat libexpat_2_5_0 \ && \ diff -r -u libexpat_2_4_9/expat/lib/ cpython-main/Modules/expat/ | tee 2-4-9.diff \ && \ diff -r -u libexpat_2_5_0/expat/lib/ scdub-expat-2-5-0/Modules/expat/ | tee 2-5-0.diff \ && \ sed -e '/^Only in /d' -e '/^\(+++\|---\) /d' -e '/^diff /d' -i 2-4-9.diff 2-5-0.diff \ && \ diff -u 2-4-9.diff 2-5-0.diff \ && \ echo 'Diff is good.'

[scdub]

@scdub I don't really understand your message. Can you maybe rephrase your question for me to better understand?

I applogize, I wasn't clear. I noticed that with 2.5.0, there is a new EXPAT_CONFIG_H definition for preventing multiple inclusions and wasn't sure if there was any changes required for pyexpatns.h or they can remain independent approaches.

[hartwork]

@scdub thanks for elaborating. The added EXPAT_CONFIG_H protection originally came up in relation to refactoring the test suite that triggered compiler warnings on redefining existing macros. I think it can safely be ignored in context of CPython.

[scdub]

@gpshead I'm not sure if this PR is of interest to you, but saw that you did initiate the work for the previous expat upgrade to 2.4.9.

[gpshead]

Confirmed that these match the libexpat 2.5.0 upstream sources from SHA256(expat-2.5.0.tar.xz)= ef2420f0232c087801abf705e89ae65f6257df6b7931d37846a193ef2e8cdcbe

with our pyexpatns.h addition and that this adds no new C APIs that need including in that "namespace" header hack.

[miss-islington]

Thanks @scdub for the PR, and @gpshead for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8, 3.9, 3.10, 3.11.
🐍🍒⛏🤖

[bedevere-bot]

GH-98784 is a backport of this pull request to the 3.11 branch.

[bedevere-bot]

GH-98785 is a backport of this pull request to the 3.10 branch.

[bedevere-bot]

GH-98786 is a backport of this pull request to the 3.9 branch.

[bedevere-bot]

GH-98787 is a backport of this pull request to the 3.8 branch.

[bedevere-bot]

GH-98788 is a backport of this pull request to the 3.7 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot aarch64 Fedora Stable Clang Installed 3.11 has failed when building commit c5f3f29.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1004/builds/393) and take a look at the build logs.
4. Check if the failure is related to this commit (c5f3f29) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1004/builds/393

Failed tests:

• test_nntplib

Failed subtests:

• setUpClass - test.test_nntplib.NetworkedNNTP_SSLTests
• test_with_statement - test.test_nntplib.NetworkedNNTP_SSLTests.test_with_statement

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

416 tests OK.

1 test failed:
test_nntplib

17 tests skipped:
test_asdl_parser test_check_c_globals test_clinic test_devpoll
test_gdb test_ioctl test_kqueue test_launcher test_msilib
test_startfile test_tix test_tk test_ttk_guionly test_winconsoleio
test_winreg test_winsound test_zipfile64

1 re-run test:
test_nntplib

Total duration: 2 min 41 sec

Click to see traceback logs

Traceback (most recent call last): File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/test/test_nntplib.py", line 252, in wrapped meth(self) File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/test/test_nntplib.py", line 298, in test_with_statementif re.search(r'(?i)KEY.TOO.SMALL', ssl_err.reason): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/re/__init__.py", line 176, in searchreturn _compile(pattern, flags).search(string) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^TypeError: expected string or bytes-like object, got 'NoneType' Traceback (most recent call last): File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/test/test_nntplib.py", line 286, in test_with_statement server =self.NNTP_CLASS(self.NNTP_HOST, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/nntplib.py", line 1028, in __init__super().__init__(host, port, user, password, readermode, File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/nntplib.py", line 337, in __init__self.sock =self._create_socket(timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/nntplib.py", line 1034, in _create_socket sock = _encrypt_on(sock, self.ssl_context, self.host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/nntplib.py", line 295, in _encrypt_onreturn context.wrap_socket(sock, server_hostname=hostname) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/ssl.py", line 517, in wrap_socketreturnself.sslsocket_class._create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/ssl.py", line 1075, in _createself.do_handshake() File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/ssl.py", line 1346, in do_handshakeself._sslobj.do_handshake() ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:992) Traceback (most recent call last): File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/multiprocessing/resource_tracker.py", line 209, in main cache[rtype].remove(name) KeyError: '/psm_d193db14' Traceback (most recent call last): File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/multiprocessing/resource_tracker.py", line 209, in main cache[rtype].remove(name) KeyError: '/psm_f4548318' Traceback (most recent call last): File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/test/test_nntplib.py", line 335, in setUpClassif re.search(r'(?i)KEY.TOO.SMALL', ssl_err.reason): ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/re/__init__.py", line 176, in searchreturn _compile(pattern, flags).search(string) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^TypeError: expected string or bytes-like object, got 'NoneType' Traceback (most recent call last): File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/multiprocessing/resource_tracker.py", line 209, in main cache[rtype].remove(name) KeyError: '/psm_5be85ea9' Traceback (most recent call last): File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/test/test_nntplib.py", line 332, in setUpClasscls.server =cls.NNTP_CLASS(cls.NNTP_HOST, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/nntplib.py", line 1028, in __init__super().__init__(host, port, user, password, readermode, File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/nntplib.py", line 337, in __init__self.sock =self._create_socket(timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/nntplib.py", line 1034, in _create_socket sock = _encrypt_on(sock, self.ssl_context, self.host) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/nntplib.py", line 295, in _encrypt_onreturn context.wrap_socket(sock, server_hostname=hostname) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/ssl.py", line 517, in wrap_socketreturnself.sslsocket_class._create( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/ssl.py", line 1075, in _createself.do_handshake() File "/home/buildbot/buildarea/3.11.cstratak-fedora-stable-aarch64.clang-installed/build/target/lib/python3.11/ssl.py", line 1346, in do_handshakeself._sslobj.do_handshake() ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:992)