Automate builds of source and docs artifacts with GitHub Actions#66
Description
This issue assumes that Release Managers are using the run_release.py script, the "current" ordering is derived from that script. Implementing the proposal below would require changes to both run_release.py and release.py. Expand the diagrams below to see a visualization of the proposal.
Current release process
- There are two stages where Sigstore is used to sign the artifacts.
- Platform experts are only tagged in once the source and docs build has been uploaded to python.org
- Source and docs builds can't verifiably originate from a specific git tag/commit.
Expand for diagram
flowchart TD pull_fork[git pull upstream -> fork] blurb[blurb+bump] tag[git tag] push_fork[git push fork] build_artifacts[Build release artifacts] gpg_and_sigstore[Sign with GPG and Sigstore] sigstore[Sign with Sigstore if needed] test_artifacts[Test release artifacts] upload_artifacts[Upload release artifacts] platform_builds[macOS and Windows builds] push_upstream[git push upstream] pull_fork --> blurb blurb --> tag tag --> build_artifacts build_artifacts --> gpg_and_sigstore gpg_and_sigstore --> test_artifacts test_artifacts --> upload_artifacts upload_artifacts --> push_fork push_fork --> platform_builds platform_builds --> sigstore sigstore --> push_upstream Proposed release process
Differences:
- Platform builds can occur concurrently to source build+test+docs.
- Source build along with testing and docs builds occur concurrently.
- Only one GPG and Sigstore signing step, at the very end in
add-to-pydotorg.py. - Would require rebuilding and repushing a new git tag+commit if a problem is detected during source/docs builds. I don't know how frequently this is an issue.
- Isolates the docs and source builds from eachother because the docs build requires many more dependencies (both from the OS package manager and from
Doc/requirements.txt).
Expand for diagram
flowchart TD pull_fork[git pull upstream -> fork] blurb[blurb+bump] tag[git tag] push_fork[git push fork] run_workflow[Run GitHub Workflow] build_artifacts[Build release artifacts] build_docs[Build docs] gpg_and_sigstore[Sign with GPG and Sigstore] test_artifacts[Test release artifacts] upload_artifacts[Upload release artifacts] platform_builds[macOS and Windows builds] push_upstream[git push upstream] pull_fork --> blurb blurb --> tag tag --> push_fork push_fork --> run_workflow push_fork --> platform_builds subgraph GitHub Actions run_workflow --> build_artifacts run_workflow --> build_docs build_artifacts --> test_artifacts build_docs end build_docs --> upload_artifacts test_artifacts --> upload_artifacts platform_builds --> upload_artifacts upload_artifacts --> gpg_and_sigstore gpg_and_sigstore --> push_upstream In order to implement the above proposal, I would do the following:
- Create a new GitHub workflow which is manually triggerable using a git commit, tag, and repo which creates source and docs tarballs as artifacts.
- Move the GPG signing step to
add-to-pydocorg.py(used byrun_release.py). Remove Sigstore signing fromrelease.py. - Create some method of detecting the source and docs tarballs by the
run_release.pyscript - Move the source/docs upload steps in
run_release.pyto be after all artifacts are ready.
After the changes have been implemented, I can modify the process for PEP 101 as necessary. I'm hoping to get some clarifications where I have things wrong from release managers, let me know!