gh-136728: Refactor build.yml CI config and multissltests.py#136729
Uh oh!
There was an error while loading. Please reload this page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -243,78 +243,32 @@ jobs: | ||
| free-threading: ${{matrix.free-threading }} | ||
| os: ${{matrix.os }} | ||
| build-ubuntu-ssltests: | ||
| name: 'Ubuntu SSL tests' | ||
| runs-on: ubuntu-24.04 | ||
| timeout-minutes: 60 | ||
| needs: build-context | ||
| if: needs.build-context.outputs.run-ubuntu == 'true' | ||
| strategy: | ||
| fail-fast: false | ||
| matrix: | ||
| include: | ||
| # Keep 1.1.1w in our list despite it being upstream EOL and otherwise | ||
| # unsupported as it most resembles other 1.1.1-work-a-like ssl APIs | ||
| # supported by important vendors such as AWS-LC. | ||
| -{ssl: openssl, ssl_ver: 1.1.1w } | ||
| -{ssl: openssl, ssl_ver: 3.0.18 } | ||
| -{ssl: openssl, ssl_ver: 3.2.6 } | ||
| -{ssl: openssl, ssl_ver: 3.3.5 } | ||
| -{ssl: openssl, ssl_ver: 3.4.3 } | ||
| -{ssl: openssl, ssl_ver: 3.5.4 } | ||
| -{ssl: awslc, ssl_ver: 1.55.0 } | ||
| # See Tools/ssl/make_ssl_data.py for notes on adding a new version | ||
| env: | ||
| SSL_VER: ${{matrix.ssl_ver }} | ||
| MULTISSL_DIR: ${{github.workspace }}/multissl | ||
| SSL_DIR: ${{github.workspace }}/multissl/${{matrix.ssl }}/${{matrix.ssl_ver }} | ||
| LD_LIBRARY_PATH: ${{github.workspace }}/multissl/${{matrix.ssl }}/${{matrix.ssl_ver }}/lib | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| @@ -328,39 +282,35 @@ jobs: | ||
| - name: Configure SSL lib env vars | ||
| run: | | ||
| echo "MULTISSL_DIR=${GITHUB_WORKSPACE}/multissl" >> "$GITHUB_ENV" | ||
| echo "SSL_DIR=${GITHUB_WORKSPACE}/multissl/${{matrix.ssl }}/${SSL_VER}" >> "$GITHUB_ENV" | ||
| echo "LD_LIBRARY_PATH=${GITHUB_WORKSPACE}/multissl/${{matrix.ssl }}/${SSL_VER}/lib" >> "$GITHUB_ENV" | ||
| - name: 'Restore SSL build' | ||
| id: cache-ssl | ||
| uses: actions/cache@v4 | ||
| with: | ||
| path: ./multissl/${{env.SSL }}/${{env.SSL_VER }} | ||
| key: ${{env.IMAGE_OS_VERSION }}-multissl-${{env.SSL }}-${{env.SSL_VER }} | ||
| - name: Install SSL | ||
| if: steps.cache-ssl.outputs.cache-hit != 'true' | ||
| run: | | ||
| python3 Tools/ssl/multissltests.py --steps=library --base-directory "$MULTISSL_DIR" --system Linux --ssl ${{matrix.ssl }} --ssl-versions ${{matrix.ssl_ver }} | ||
hugovk marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading. Please reload this page. | ||
| - name: Add ccache to PATH | ||
| run: | | ||
| echo "PATH=/usr/lib/ccache:$PATH" >> "$GITHUB_ENV" | ||
| - name: Configure CPython | ||
| run: | | ||
| CMD=(./configure CFLAGS="-fdiagnostics-format=json" --config-cache --enable-slower-safety --with-pydebug --with-openssl="$SSL_DIR") | ||
| if [ "${{matrix.ssl }}" = "openssl" ]; then | ||
| "${CMD[@]}" | ||
| else | ||
| "${CMD[@]}" --with-builtin-hashlib-hashes=blake2 --with-ssl-default-suites=openssl | ||
Member There was a problem hiding this comment. Can we populate a variable called "CONFIGURE_FLAGS" instead and use a switch/case so that we exactly match the SSL libname. It will become easier in the future if we add BoringSSL or LibreSSL tests. Member There was a problem hiding this comment. At this point I'd almost suggest using a small Python script to calculate flags. We should avoid complex shell logic in workflow files as much as possible, I don't want to read a bash switch-case in YAML! An if-statement is simpler to understand here for me, though the Member There was a problem hiding this comment. Oh yeah, a small Python script is also fine; here what matters to me is to be able to easily add a new library without having 3km long of options. As for | ||
| fi | ||
| - name: Build CPython | ||
| run: make -j4 | ||
picnixz marked this conversation as resolved. Show resolvedHide resolvedUh oh!There was an error while loading. Please reload this page. | ||
| - name: Display build info | ||
| run: make pythoninfo | ||
| - name: Verify python is linked to AWS-LC | ||
| if: matrix.ssl == 'aws-lc' | ||
Comment on lines 312 to +313 Member There was a problem hiding this comment. We should, if possible, do it for OpenSSL as well. | ||
| run: ./python -c 'import ssl; print(ssl.OPENSSL_VERSION)' | grep AWS-LC | ||
| - name: SSL tests | ||
| run: ./python Lib/test/ssltests.py | ||
| @@ -446,7 +396,7 @@ jobs: | ||
| key: ${{runner.os }}-multissl-openssl-${{env.OPENSSL_VER }} | ||
| - name: Install OpenSSL | ||
| if: steps.cache-openssl.outputs.cache-hit != 'true' | ||
| run: python3 Tools/ssl/multissltests.py --steps=library --base-directory "$MULTISSL_DIR" --ssl 'openssl' --ssl-versions "$OPENSSL_VER" --system Linux | ||
| - name: Add ccache to PATH | ||
| run: | | ||
| echo "PATH=/usr/lib/ccache:$PATH" >> "$GITHUB_ENV" | ||
| @@ -564,7 +514,7 @@ jobs: | ||
| key: ${{matrix.os }}-multissl-openssl-${{env.OPENSSL_VER }} | ||
| - name: Install OpenSSL | ||
| if: steps.cache-openssl.outputs.cache-hit != 'true' | ||
Member There was a problem hiding this comment. With | ||
| run: python3 Tools/ssl/multissltests.py --steps=library --base-directory "$MULTISSL_DIR" --ssl 'openssl' --ssl-versions "$OPENSSL_VER" --system Linux | ||
| - name: Add ccache to PATH | ||
| run: | | ||
| echo "PATH=/usr/lib/ccache:$PATH" >> "$GITHUB_ENV" | ||
| @@ -689,9 +639,7 @@ jobs: | ||
| - build-windows-msi | ||
| - build-macos | ||
| - build-ubuntu | ||
| - build-ubuntu-ssltests | ||
| - build-ios | ||
| - build-wasi | ||
| - test-hypothesis | ||
| @@ -706,9 +654,9 @@ jobs: | ||
| uses: re-actors/alls-green@05ac9388f0aebcb5727afa17fcccfecd6f8ec5fe | ||
| with: | ||
| allowed-failures: >- | ||
| build-android, | ||
| build-windows-msi, | ||
| build-ubuntu-ssltests, | ||
| test-hypothesis, | ||
| cifuzz, | ||
| allowed-skips: >- | ||
| @@ -728,8 +676,8 @@ jobs: | ||
| !fromJSON(needs.build-context.outputs.run-ubuntu) | ||
| && ' | ||
| build-ubuntu, | ||
| build-ubuntu-ssltests, | ||
| build-wasi, | ||
| test-hypothesis, | ||
| build-asan, | ||
| build-san, | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -22,7 +22,4 @@ jobs: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| persist-credentials: false | ||
| - uses: j178/prek-action@v1 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Oops, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.