Skip to content

codrsquad/portable-python

Repository files navigation

Portable python binaries

Version on pypiTested with Github ActionsTest coveragePython versions tested (link to github project)

Portable-Python is a CLI (and a python library) for compiling python binaries from source than can be decompressed in any folder, and used from there without further ado (ie: no need to run an "installer").

Motivation

The idea here is to allow for automated systems to:

  • Easily obtain a python binary, that can be used in sandboxes / workstations / laptops / instances...
  • Inspect any python installation, and point out how portable it is, which shared or non-standard libraries it is using

Installation

portable-python is a regular python CLI, it can be installed with:

pickley:

pickley install portable-python portable-python --help portable-python inspect /usr/bin/python3 

Or pipx:

pipx install portable-python portable-python inspect /usr/bin/python3 

Using pip install (a CI builder would probably do this):

/usr/bin/python3 -mvenv /tmp/pp /tmp/pp/bin/python -mpip install portable-python /tmp/pp/bin/portable-python --help /tmp/pp/bin/portable-python inspect /usr/bin/python3 

Supported operating systems

Portable python binaries can be built for Linux and MacOS (Intel/M1/M2).

Currently Windows is NOT supported, contributions are welcome.

Python binaries can be produced as "portable" (statically linked, can run from any folder where the binary is unpacked in), or with a --prefix (build targeted to live in a pre-determined folder, like /apps/pythonM.m)

Operating systemPortable--prefix
Linux
Macos
Windows

Building a portable cpython

Once portable-python is installed:

Build a binary:

cd some-temp-folder portable-python build 3.9.7 ls -l dist/cpython-3.9.7-macos-arm64.tar.gz 

Unpack it somewhere:

tar -C ~/tmp/versions/ -xf dist/cpython-3.9.7-macos-arm64.tar.gz ls -l ~/tmp/versions/ 

It's ready to be used:

~/tmp/versions/3.9.7/bin/python --version 

Note that you can use --dryrun mode to inspect what would be done without doing it:

$ portable-python --dryrun build 3.9.7 INFO selected: xz openssl gdbm (3 modules) xz:5.2.5 openssl:1.1.1k gdbm:1.18.1 INFO Platform: macos-x86_64 ... -------------- -- xz:5.2.5 -- -------------- Would download https://tukaani.org/xz/xz-5.2.5.tar.gz Would untar build/sources/xz-5.2.5.tar.gz -> build/components/xz INFO env PATH=build/deps/bin:/usr/bin:/bin INFO env MACOSX_DEPLOYMENT_TARGET=10.14 Would run: ./configure --prefix=build/deps --enable-shared=no --enable-static=yes ... ... ------------------- -- cpython:3.9.7 -- ------------------- Would download https://www.python.org/ftp/python/3.9.7/Python-3.9.7.tar.xz Would untar build/sources/Python-3.9.7.tar.xz -> build/components/cpython ... Would run: ./configure --prefix=/ppp-marker/3.9.7 --enable-optimizations ... Would run: /usr/bin/make Would run: /usr/bin/make install DESTDIR=build ... Would tar build/3.9.7 -> dist/cpython-3.9.7-macos-x86_64.tar.gz 

Library

Portable Python can be used as a python library to invoke builds, or inspect an installation.

Invoke a build from python code:

from portable_python import BuildSetup setup = BuildSetup("cpython:3.9.7") setup.compile() 

Invoke an inspection from python code:

from portable_python.inspector import PythonInspector inspector = PythonInspector("/usr/bin/python3") print(inspector.represented()) problem = inspector.full_so_report.get_problem(portable=True) if problem: print("oops, it is not portable!: %s" % problem) 

From source, contributions welcome!:

git clone https://github.com/codrsquad/portable-python.git cd portable-python python3 -mvenv .venv .venv/bin/pip install -r requirements.txt -r tests/requirements.txt .venv/bin/pip install -e . .venv/bin/portable-python --help .venv/bin/portable-python inspect /usr/bin/python3 tox -e py311 tox -e style 

Build folder structure

portable-python uses this file structure (build/ and dist/ folders configurable):

build/ ppp-marker/3.9.7/ # Full installation (after build completes) components/ # Builds of statically compiled extension modules are here deps/ # --prefix=.../deps passed to all component ./configure scripts sources/ openssl-1.1.1k.tar.gz # Downloaded artifacts (downloaded only once) dist/ cpython-3.9.7-macos-arm64.tar.gz # Ready-to-go portable binary tarball 

Guiding principles

  • Focuses on just one thing: compile a portable python, and validate that it is indeed portable, produce outcome in (configurable) ./dist/ folder and that's it
  • No patches: C compilation is done as simply as possible without modifying the upstream source code. Rely solely on the make/configure scripts, typically via stuff like --enable-shared=no
  • Builds are validated, an important part of the effort was to write up code that is able to inspect a python installation and detect whether it is portable or not (and why not if so).
  • Only the last few non-EOL versions of python are supported (no historical stuff)
  • As time goes on, the code of this tool will evolve so that the latest pythons keep building (but won't worry that older versions still keep building)

For this repo itself:

  • Code is pure python, it is a CLI with one entry-point called portable-python
    • Can be ran in a debugger
    • 100% test coverage, has a --dryrun mode to help with testing / debugging / seeing what would be done quickly
    • No shell scripts (those are hard to maintain/test/debug)
    • Can be pip install-ed and reused

About

Portable python binaries

Resources

License

Security policy

Stars

Watchers

Forks

Packages

No packages published

Contributors 8

Languages