Unexpected SystemError in Python3.12.0a7: `<method 'startswith' of 'str' objects> returned a result with an exception set`

[abravalheri]

Bug report

I noticed recently that Setuptools CI started failing for Python 3.12.0a7 with a SystemError that is difficult to understand:

SystemError: <built-in method startswith of str object at 0x7fe399991298> returned a result with an exception set

At a first glance this error does not make much sense because str.startswith is a built-in method in a built-in data structure. I am struggling to see how it can have an exception set attached to the result.

I managed to create the following simplified reproducer:

docker run --rm -it python:3.12.0a7-bullseye
Python 3.12.0a7 (main, Apr 12 2023, 14:13:09) [GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> class Distribution:
... @property
... def version(self):
... try:
... return self._version
... except AttributeError as e:
... # ... complex code omitted for the sake of simplification ...
... raise ValueError("Missing Version")
... def __getattr__(self, attr):
... if attr.startswith("_"):
... raise AttributeError(attr)
... # ... complex code omitted for the sake of simplification ...
... return 42
...
>>> dist = Distribution()
>>> dist.version
Traceback (most recent call last):
 File "<stdin>", line 5, in version
 File "<stdin>", line 11, in __getattr__
AttributeError: _version. Did you mean: 'version'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "<stdin>", line 8, in version
ValueError: Missing Version

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 File "<stdin>", line 10, in __getattr__
SystemError: <method 'startswith' of 'str' objects> returned a result with an exception set
>>>

The error seems to be related to the combination of the following factors:

1. A property is being used¹
2. The property getter access an undefined attribute and triggers __getattr__
3. __getattr__ raises an AttributeError
4. The property getter is supposed to catch the AttributeError exception and raise a different exeception.

My expectation is that the SystemError never gets triggered with the confusing message. Instead the example should end up with a ValueError.

Please note that the example is very simplified. For the realistic implementation please consider pkg_resources.Distribution.

Your environment

• CPython versions tested on: 3.12.0a7
• Operating system and architecture:
  Reproducer tested on both Ubuntu 20.04.6 LTS machine and python:3.12.0a7-bullseye container

Problem also identified in GitHub Actions runners: ubuntu-latest, macos-latest, windows-latest (see https://github.com/pypa/setuptools/actions/runs/4710601389?pr=3893) for Python 3.12-dev.

Footnotes

1. If we replace the property with a regular method, the example works as expected. ↩

[sunmy2019]

It should be fixed by #103332

[sobolevn]

I agree that this should be fixed in main. Please, reopen if it is not fixed.

[abravalheri]

Thank you very much.

I run the test above with an adapted Dockerfile that gets the latest `main` and it seems to work

FROM buildpack-deps:bullseye

# ensure local python is preferred over distribution python
ENV PATH /usr/local/bin:$PATH

# http://bugs.python.org/issue19846
# > At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK.
ENV LANG C.UTF-8

# runtime dependencies
RUN set -eux; \
 apt-get update; \
 apt-get install -y --no-install-recommends \
 libbluetooth-dev \
 tk-dev \
 uuid-dev \
 ; \
 rm -rf /var/lib/apt/lists/*

ENV PYTHON_VERSION 3.12.0dev+3e0fec7


RUN set -eux; \
 \
 wget -O /tmp/python.tar.gz "https://github.com/python/cpython/tarball/3e0fec7e07a71bdeeab7554e980110fbc47763b9"; \
 mkdir -p /usr/src/python; \
 tar --extract --directory /usr/src/python --strip-components=1 --file /tmp/python.tar.gz; \
 rm /tmp/python.tar.gz; \
 \
 cd /usr/src/python; \
 gnuArch="$(dpkg-architecture --query DEB_BUILD_GNU_TYPE)"; \
 ./configure \
 --build="$gnuArch" \
 --enable-loadable-sqlite-extensions \
 --enable-optimizations \
 --enable-option-checking=fatal \
 --enable-shared \
 --with-lto \
 --with-system-expat \
 --without-ensurepip \
 ; \
 nproc="$(nproc)"; \
 EXTRA_CFLAGS="$(dpkg-buildflags --get CFLAGS)"; \
 LDFLAGS="$(dpkg-buildflags --get LDFLAGS)"; \
 make -j "$nproc" \
 "EXTRA_CFLAGS=${EXTRA_CFLAGS:-}" \
 "LDFLAGS=${LDFLAGS:-}" \
 "PROFILE_TASK=${PROFILE_TASK:-}" \
 ; \
# https://github.com/docker-library/python/issues/784
# prevent accidental usage of a system installed libpython of the same version
 rm python; \
 make -j "$nproc" \
 "EXTRA_CFLAGS=${EXTRA_CFLAGS:-}" \
 "LDFLAGS=${LDFLAGS:--Wl},-rpath='\$\$ORIGIN/../lib'" \
 "PROFILE_TASK=${PROFILE_TASK:-}" \
 python \
 ; \
 make install; \
 \
# enable GDB to load debugging data: https://github.com/docker-library/python/pull/701
 bin="$(readlink -ve /usr/local/bin/python3)"; \
 dir="$(dirname "$bin")"; \
 mkdir -p "/usr/share/gdb/auto-load/$dir"; \
 cp -vL Tools/gdb/libpython.py "/usr/share/gdb/auto-load/$bin-gdb.py"; \
 \
 cd /; \
 rm -rf /usr/src/python; \
 \
 find /usr/local -depth \
 \( \
 \( -type d -a \( -name test -o -name tests -o -name idle_test \) \) \
 -o \( -type f -a \( -name '*.pyc' -o -name '*.pyo' -o -name 'libpython*.a' \) \) \
 \) -exec rm -rf '{}' + \
 ; \
 \
 ldconfig; \
 \
 python3 --version

CMD ["python3"]

Ref: https://github.com/docker-library/python/blob/cb6ecdbcbc7e30a180481a9911f5f1f470b9dba1/3.12-rc/bullseye/Dockerfile