Skip to content

gh-202: Simplify Windows installation#203

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jkloth wants to merge 1 commit into
base:main
Choose a base branch
from
Open

gh-202: Simplify Windows installation #203

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 0 additions & 21 deletions doc/usage.rst
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -23,27 +23,6 @@ extension. Commands on Fedora to install dependencies:
* Python 3: ``sudo dnf install python3-devel``
* PyPy: ``sudo dnf install pypy-devel``

Windows notes
-------------

On Windows, to allow pyperformance to build dependencies from source
like ``greenlet``, ``dulwich`` or ``psutil``, if you want to use a
``python.exe`` built from source, you should not use the ``python.exe``
directly. Instead, you must run the little-known command ``PC\layout``
to create a filesystem layout that resembles an installed Python::

.\python.bat -m PC.layout --preset-default --copy installed -v

(Use the ``--help`` flag for more info about ``PC\layout``.)

Now you can use the "installed" Python executable::

installed\python.exe -m pip install pyperformance
installed\python.exe -m pyperformance run ...

Using an *actually* installed Python executable (e.g. via ``py``)
works fine too.

Comment on lines -26 to -46
Copy link
Member

@ericsnowcurrentlyericsnowcurrentlyMay 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like this should remain for the sake of Python versions that don't have the fix. So a note should be added identifying the last Python version to which this applies.

Copy link
Author

@jklothjklothMay 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was going with the assumption that using a source build on Windows is such a rare occurrence that this would be in the realm of CPython developers already operating out of a GH clone. This would imply an up to date (-ish) checkout, therefore containing the bug fix.

Copy link
Member

@ericsnowcurrentlyericsnowcurrentlyMay 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, that makes sense. The catch is that we run benchmarks against older CPython versions too.

Copy link
Author

@jklothjklothMay 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do note, though, the removed verbiage only applies to source builds. Other means of running Python (nuget, winget, installer) work as is. I would say workflows that clone a released Python branch and build it to run pyperformance would be better served using the released binary for the comparison.

For example:

nuget.exe install python -Version 3.10 [-OutputDirectory <dir>]

will download and unpack the released distribution for 3.10.0 to <dir>\python as a full usable interpreter (<dir>\python\tools\python.exe). This already is done in the build scripts just to build the interpreter now[1].
nuget.exe is also available in the <srcdir>\externals directory (see [1]). This has the added benefit of guaranteeing isolation from previous builds.

[1] Unless there is an existing Python installation usable for building.

Copy link
Author

@jklothjklothMay 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll admit, however, I have no idea how any current workflows using pyperformance on Windows are implemented. Although the current guidance seems to be quite heavy already.

Copy link
Member

@gvanrossumgvanrossumMay 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very few have managed to successfully run PyPerformance on Windows, because it has been so broken for so long (and the test suite is still broken -- as soon as you start it, the process detaches and runs in the background, which I didn't even know was possible in DOS Windows :-).

I have a reason for building past versions from source though: You never know which aspects of your current setup (compiler version, Windows version, hardware, version of other tooling installed) might affect the performance of the binary. It wouldn't be the first time that someone reported a significant but mysterious/irreproducible speedup or slowdown between two releases only to find out that they were comparing binaries built with different compiler versions. If the goal is to measure the effect of the source code changes rather than simply the progress in compiler technology, it's important to try to keep as many variables unchanged as possible.


Run benchmarks
==============
Expand Down
6 changes: 6 additions & 0 deletions pyperformance/venv.py
View file
Open in desktop
Original file line numberDiff line numberDiff line change
Expand Up@@ -137,6 +137,12 @@ def _get_envvars(inherit=None, osname=None):
copy_env.extend(inherit)

env ={}
# Recent setuptools versions have a bug that prevents being
# used to compile from a source build virtual environment.
# [BUG](https://github.com/pypa/setuptools/issues/3325)
# The stdlib distutils does not have this problem.
Copy link
Member

@ericsnowcurrentlyericsnowcurrentlyMay 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when the stdlib distutils goes away?

Copy link
Member

@gvanrossumgvanrossumMay 24, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Presumably the bug would have to be fixed in setuptools?

if os.name == 'nt':
env['SETUPTOOLS_USE_DISTUTILS'] = 'stdlib'
for name in copy_env:
if name in os.environ:
env[name] = os.environ[name]
Expand Down