setup.py grep_headers_for() is broken by design

[tiran]

BPO	45598
Nosy	@amauryfa, @abalkin, @tiran, @ned-deily, @merwok, @meadori, @thesamesam
PRs	bpo-45598: Remove ffi header header #29352

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-10-24.12:05:01.584>
labels = ['type-bug', '3.9', '3.10', '3.11', 'ctypes', 'build']
title = 'setup.py grep_headers_for() is broken by design'
updated_at = <Date 2021-11-03.03:37:49.619>
user = 'https://github.com/tiran'

bugs.python.org fields:

activity = <Date 2021-11-03.03:37:49.619>
actor = 'ned.deily'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build', 'ctypes']
creation = <Date 2021-10-24.12:05:01.584>
creator = 'christian.heimes'
dependencies = []
files = []
hgrepos = []
issue_num = 45598
keywords = ['patch']
message_count = 4.0
messages = ['404926', '404927', '405379', '405580']
nosy_count = 7.0
nosy_names = ['amaury.forgeotdarc', 'belopolsky', 'christian.heimes', 'ned.deily', 'eric.araujo', 'meador.inge', 'thesamesam']
pr_nums = ['29352']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue45598'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

[tiran]

The setup.py helper function grep_headers_for() is used by ctypes build step to search header files for functions. The function kinda works on most platforms, but only by accident. On my system it opens all header files that match /usr/include/*.h and returns true if any (!) header file contains the string (!) "ffi_prep_cif_var". The check would also match "ffi_prep_cif_var" in a comment of an unrelated header file.

We cannot limit the search to "ffi.h" only, because that would break multiarch builds with a header file like this:

 #if defined(__i386__)
 #include "ffi-i386.h"
 #elif defined(__powerpc64__)
 #include "ffi-ppc64.h"
 ...

Solutions:

1. Use "AC_EGREP_HEADER" autoconf macro. It runs a header file through the C preprocessor
2. Use AC_LINK_IFELSE autoconf macro. This macro compiles and links code to detect if a library exports a function.
3. Re-implement any of the macros in pure Python (ugh!)
4. Just assume that libffi supports ffi_prep_cif_var, ffi_prep_closure_loc, and ffi_closure_alloc on our target platforms. According to [1] the functions have been around for over a decade.

I favor (4).

Ned, what about USING_APPLE_OS_LIBFFI? Is it still relevant?

[1] https://github.com/libffi/libffi/blame/48bdb02867edb7e9f3785ccb4bdff1087fb44246/include/ffi.h.in#L309

[tiran]

If we remove grep_headers_for(), then we can also close bpo-42325

[merwok]

We could look in distro patch trackers to see they patch CPython’s setup.py to evaluate the impact of removing this function.