Skip to content

Conversation

@encukou
Copy link
Member

@encukouencukou commented May 24, 2021

A previous commit broke a check in sysconfig when building cpython itself.
This caused builds of the standard library modules to search a wrong
location (the installed location rather than the source directory) for
header files with the net effect that a make install
incorrectly caused all extension modules to be rebuilt again and
with incorrect include file paths.

There's a complication. When building Python, we need two distinct
"include" directories:

  • source .h files
  • install target for .h files

Note that this doesn't matter except when building Python from source.

Historically:

  • source .h files were in the distutils scheme under 'include'
  • the install directory was in the distutils.command.install scheme
    under 'headers'

GH-24549 merged these; sysconfig is now the single source of truth and
distutils is derived from it.

Introduce a "secret" scheme path, 'headers', which contains
the install target. It is only present when building Python.
The distutils code uses it if present, and falls back to 'include'.

https://bugs.python.org/issue41282

ned-deilyand others added 2 commits May 24, 2021 03:53
A previous commit broke a check in sysconfig when building cpython itself. This caused builds of the standard library modules to search a wrong location (the installed location rather than the source directory) for header files with the net effect that a ``make install`` incorrectly caused all extension modules to be rebuilt again and with incorrect include file paths.
When building Python, we need two distinct "include" directories: - source .h files - install target for .h files Note that this doesn't matter except when building Python from source. Historically: - source .h files were in the distutils scheme under 'include' - the install directory was in the distutils.command.install scheme under 'headers' pythonGH-24549 merged these; sysconfig is now the single source of truth and distutils is derived from it. This commit introduces a "secret" scheme path, 'headers', which contains the install target. It is only present when building Python. The distutils code uses it if present, and falls back to 'include'.
@encukouencukou added the needs backport to 3.10 only security fixes label May 24, 2021
@encukouencukou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 24, 2021
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @encukou for commit b300aac 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

@bedevere-botbedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 24, 2021
@frenzymadness
Copy link
Contributor

Your analysis makes sense, the code looks good and I'm able to identify why I've never met this problem before when I was preparing the merge of sysconfig modules.

I'm sorry for the mistake I made.

Copy link
Member

@ned-deilyned-deily left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me and a more robust one than my initial stab at it. Thanks, @encukou! If you don't get to it soonish, I'll probably merge this to get the backport to 3.10 going.

@miss-islington
Copy link
Contributor

Thanks @encukou for the PR, and @ned-deily for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 24, 2021
A previous commit broke a check in sysconfig when building cpython itself. This caused builds of the standard library modules to search a wrong location (the installed location rather than the source directory) for header files with the net effect that a ``make install`` incorrectly caused all extension modules to be rebuilt again and with incorrect include file paths. When building Python, we need two distinct "include" directories: - source .h files - install target for .h files Note that this doesn't matter except when building Python from source. Historically: - source .h files were in the distutils scheme under 'include' - the install directory was in the distutils.command.install scheme under 'headers' pythonGH-24549 merged these; sysconfig is now the single source of truth and distutils is derived from it. This commit introduces a "secret" scheme path, 'headers', which contains the install target. It is only present when building Python. The distutils code uses it if present, and falls back to 'include'. Co-authored-by: Ned Deily <[email protected]> (cherry picked from commit 563bd5a) Co-authored-by: Petr Viktorin <[email protected]>
@bedevere-bot
Copy link

GH-26336 is a backport of this pull request to the 3.10 branch.

@bedevere-botbedevere-bot removed the needs backport to 3.10 only security fixes label May 24, 2021
miss-islington added a commit that referenced this pull request May 24, 2021
A previous commit broke a check in sysconfig when building cpython itself. This caused builds of the standard library modules to search a wrong location (the installed location rather than the source directory) for header files with the net effect that a ``make install`` incorrectly caused all extension modules to be rebuilt again and with incorrect include file paths. When building Python, we need two distinct "include" directories: - source .h files - install target for .h files Note that this doesn't matter except when building Python from source. Historically: - source .h files were in the distutils scheme under 'include' - the install directory was in the distutils.command.install scheme under 'headers' GH-24549 merged these; sysconfig is now the single source of truth and distutils is derived from it. This commit introduces a "secret" scheme path, 'headers', which contains the install target. It is only present when building Python. The distutils code uses it if present, and falls back to 'include'. Co-authored-by: Ned Deily <[email protected]> (cherry picked from commit 563bd5a) Co-authored-by: Petr Viktorin <[email protected]>
@encukouencukou deleted the sysconfig-pr26327 branch May 25, 2021 08:51
@jaracojaraco mentioned this pull request May 9, 2022
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants

@encukou@bedevere-bot@frenzymadness@miss-islington@ned-deily@the-knights-who-say-ni