test_bdb and test_distutils fail on installed Python 3.9, 3.10 and 3.11 (setuptools 60.9.3, pip 22.0.4)

[vstinner]

BPO	47013
Nosy	@jaraco, @vstinner, @ned-deily

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2022-03-14.16:14:07.532>
labels = ['3.8', '3.9', '3.10', '3.11', '3.7', 'tests']
title = 'test_bdb and test_distutils fail on installed Python 3.9, 3.10 and 3.11 (setuptools 60.9.3, pip 22.0.4)'
updated_at = <Date 2022-03-16.01:15:39.379>
user = 'https://github.com/vstinner'

bugs.python.org fields:

activity = <Date 2022-03-16.01:15:39.379>
actor = 'jaraco'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Tests']
creation = <Date 2022-03-14.16:14:07.532>
creator = 'vstinner'
dependencies = []
files = []
hgrepos = []
issue_num = 47013
keywords = []
message_count = 8.0
messages = ['415157', '415158', '415160', '415161', '415162', '415164', '415180', '415311']
nosy_count = 3.0
nosy_names = ['jaraco', 'vstinner', 'ned.deily']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue47013'
versions = ['Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10', 'Python 3.11']

[vstinner]

Since build 298, test_bdb test_distutils fail on s390x Fedora Rawhide Clang Installed 3.10:
https://buildbot.python.org/all/#/builders/694/builds/298

2 changes of build 298:

• bpo-46986: Upgrade bundled setuptools to 60.9.3 (bpo-46986: Upgrade bundled setuptools to 60.9.3 #31820)(21 hours ago)
• [3.10] bpo-46985: Upgrade bundled pip to 22.0.4 (bpo-46985: Upgrade bundled pip to 22.0.4 #31819) ([3.10] bpo-46985: Upgrade bundled pip to 22.0.4 (GH-31819) #31849)(21 hours ago)

Logs:

0:02:03 load avg: 9.95 [213/427/1] test_distutils failed (uncaught exception) -- running: test_concurrent_futures (42.4 sec)
test test_distutils crashed -- Traceback (most recent call last):
 File "/home/dje/cpython-buildarea/3.10.edelsohn-fedora-rawhide-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 335, in _runtest_inner
 refleak = _runtest_inner2(ns, test_name)
 File "/home/dje/cpython-buildarea/3.10.edelsohn-fedora-rawhide-z.clang-installed/build/target/lib/python3.10/test/libregrtest/runtest.py", line 280, in _runtest_inner2
 the_module = importlib.import_module(abstest)
 File "/home/dje/cpython-buildarea/3.10.edelsohn-fedora-rawhide-z.clang-installed/build/target/lib/python3.10/importlib/__init__.py", line 126, in import_module
 return _bootstrap._gcd_import(name[level:], package, level)
 File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
 File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
 File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
 File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
 File "<frozen importlib._bootstrap_external>", line 883, in exec_module
 File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
 File "/home/dje/cpython-buildarea/3.10.edelsohn-fedora-rawhide-z.clang-installed/build/target/lib/python3.10/test/test_distutils.py", line 15, in <module>
 import distutils.tests
ModuleNotFoundError: No module named 'distutils.tests'

======================================================================
FAIL: test_skip (test.test_bdb.StateTestCase)
----------------------------------------------------------------------

Traceback (most recent call last):
 File "/home/dje/cpython-buildarea/3.10.edelsohn-fedora-rawhide-z.clang-installed/build/target/lib/python3.10/test/test_bdb.py", line 730, in test_skip
 with TracerRun(self, skip=skip) as tracer:
 File "/home/dje/cpython-buildarea/3.10.edelsohn-fedora-rawhide-z.clang-installed/build/target/lib/python3.10/test/test_bdb.py", line 448, in __exit__
 self.test_case.fail(err_msg)
 File "/home/dje/cpython-buildarea/3.10.edelsohn-fedora-rawhide-z.clang-installed/build/target/lib/python3.10/test/test_bdb.py", line 582, in fail
 raise self.failureException(msg) from None
AssertionError: Wrong event type at expect_set item 2, got 'call'
 Expected: ('line', 3, 'tfunc_import')
 Got: ('call', 84, 'find_spec'), ('quit',),

[vstinner]

Reproduce:

$ ./configure --prefix /opt/py310 && make clean && make && make install 
$ cd /somewhere/else/
$ /opt/py310/bin/python3 -m test -v test_bdb
...
FAIL: test_skip (test.test_bdb.StateTestCase)
...
$ /opt/py310/bin/python3 -m test -v test_distutils 
...
ModuleNotFoundError: No module named 'distutils.tests'
...

setuptools installs this file:

$ cat /opt/py310/lib/python3.10/site-packages/distutils-precedence.pth
import os; var = 'SETUPTOOLS_USE_DISTUTILS'; enabled = os.environ.get(var, 'local') == 'local'; enabled and __import__('_distutils_hack').add_shim();

[vstinner]

See: [BUG] Having setuptools installed causes cpython stdlib build to fail
pypa/setuptools#3007

[vstinner]

x86 Gentoo Installed with X 3.9:
https://buildbot.python.org/all/#builders/527/builds/474

4 tests failed:
test_bdb test_distutils test_importlib test_peg_generator

Python 3.9 gets two more issues: test_importlib and test_peg_generator.

======================================================================
FAIL: test_package_discovery (test.test_importlib.test_main.DiscoveryTests)
----------------------------------------------------------------------

Traceback (most recent call last):
 File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/test_importlib/test_main.py", line 159, in test_package_discovery
 assert all(
AssertionError

0:06:27 load avg: 3.96 [ 83/425/3] test_peg_generator failed (uncaught exception) -- running: test_multiprocessing_forkserver (1 min 27 sec)
Warning -- warnings.filters was modified by test_peg_generator
Failed to import test module: test.test_peg_generator.test_c_parser
Traceback (most recent call last):
 File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
 module = self._get_module_from_name(name)
 File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
 __import__(name)
 File "/buildbot/buildarea/cpython/3.9.ware-gentoo-x86.installed/build/target/lib/python3.9/test/test_peg_generator/test_c_parser.py", line 4, in <module>
 from distutils.tests.support import TempdirManager
ModuleNotFoundError: No module named 'distutils.tests'

[vstinner]

AMD64 Fedora Stable Clang Installed 3.x:
https://buildbot.python.org/all/#builders/350/builds/1621

2 re-run tests:
test_bdb test_distutils