gh-103053: Fix make check-clean-src: check "python" program

[vstinner]

"make check-clean-src" now also checks if the "python" program is found in the source directory: fail with an error if it does exist.

• Issue: test_tools: test_freeze_simple_script() times out after 15 min: --with-lto --enable-optimizations (s390x LTO buildbots) #103053

[miss-islington]

Thanks @vstinner for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

[bedevere-app]

GH-110453 is a backport of this pull request to the 3.12 branch.

[bedevere-app]

GH-110454 is a backport of this pull request to the 3.11 branch.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Debian 3.x has failed when building commit a155f9f.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/49/builds/6810) and take a look at the build logs.
4. Check if the failure is related to this commit (a155f9f) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/49/builds/6810

Failed tests:

• test_tools

Failed subtests:

• test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_w8i674vz/tmpwqpxzm08/python-build', '-j2']' returned non-zero exit status 2. Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.x.edelsohn-debian-z/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_xsrn4bbi/tmp2i6t5cmq/python-build', '-j2']' returned non-zero exit status 2.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x SLES 3.x has failed when building commit a155f9f.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/540/builds/6775) and take a look at the build logs.
4. Check if the failure is related to this commit (a155f9f) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/540/builds/6775

Failed tests:

• test_tools

Failed subtests:

• test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_knw1itkv/tmpo5ggq57c/python-build', '-j2']' returned non-zero exit status 2. Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_w5_da6bk/tmp0zwj8_94/python-build', '-j2']' returned non-zero exit status 2.

[vstinner]

cc @erlend-aasland: Another build fix ;-)

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Debian 3.11 has failed when building commit f7a1d7d.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/944/builds/1290) and take a look at the build logs.
4. Check if the failure is related to this commit (f7a1d7d) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/944/builds/1290

Failed tests:

• test_tools

Failed subtests:

• test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-debian-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-debian-z/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.11.edelsohn-debian-z/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-debian-z/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmp0rafp67o/python-build', '-j2']' returned non-zero exit status 2. Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-debian-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-debian-z/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.11.edelsohn-debian-z/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-debian-z/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_icty5ucf/tmprcond1ap/python-build', '-j2']' returned non-zero exit status 2.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x SLES 3.12 has failed when building commit 67028f0.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1129/builds/402) and take a look at the build logs.
4. Check if the failure is related to this commit (67028f0) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1129/builds/402

Failed tests:

• test_tools

Failed subtests:

• test_create_connection_local_addr_skip_different_family - test.test_asyncio.test_events.SelectEventLoopTests.test_create_connection_local_addr_skip_different_family
• test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.12.edelsohn-sles-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.12.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.12.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.12.edelsohn-sles-z/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_30mjw648/tmp_528u491/python-build', '-j2']' returned non-zero exit status 2. Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.12.edelsohn-sles-z/build/Lib/test/test_asyncio/test_events.py", line 695, in test_create_connection_local_addr_skip_different_familywithself.assertRaises(OSError): AssertionError: OSError not raised Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.12.edelsohn-sles-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.12.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.12.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.12.edelsohn-sles-z/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_2krhochy/tmpkztlz7w_/python-build', '-j2']' returned non-zero exit status 2.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL7 3.11 has failed when building commit f7a1d7d.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/1033/builds/1081) and take a look at the build logs.
4. Check if the failure is related to this commit (f7a1d7d) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/1033/builds/1081

Failed tests:

• test_tools

Failed subtests:

• test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpsyhm1gh9/python-build', '-j2']' returned non-zero exit status 2. Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_jjzl5qyt/tmp0_ifcbt9/python-build', '-j2']' returned non-zero exit status 2.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x Fedora LTO 3.11 has failed when building commit f7a1d7d.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/975/builds/889) and take a look at the build logs.
4. Check if the failure is related to this commit (f7a1d7d) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/975/builds/889

Failed tests:

• test_tools

Failed subtests:

• test_pipesize_default - test.test_subprocess.ProcessTestCaseNoPoll.test_pipesize_default
• test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-fedora-z.lto/build/Lib/test/test_subprocess.py", line 761, in test_pipesize_defaultself.assertEqual( AssertionError: 65536 != 8192 Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-fedora-z.lto/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-fedora-z.lto/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.11.edelsohn-fedora-z.lto/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-fedora-z.lto/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpp2n37l22/python-build', '-j2']' returned non-zero exit status 2. Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-fedora-z.lto/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-fedora-z.lto/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.11.edelsohn-fedora-z.lto/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-fedora-z.lto/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python_pk93x51v/tmputxhy5gk/python-build', '-j2']' returned non-zero exit status 2.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL7 LTO 3.11 has failed when building commit f7a1d7d.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/924/builds/1094) and take a look at the build logs.
4. Check if the failure is related to this commit (f7a1d7d) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/924/builds/1094

Failed tests:

• test_tools

Failed subtests:

• test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/test_python__qtwyhf8/tmpmhg0r3kf/python-build', '-j2']' returned non-zero exit status 2. Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script outdir, scriptfile, python = helper.prepare(script, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 147, in prepare _run_quiet([MAKE, '-C', builddir, parallel]) File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 33, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpdugc15h2/python-build', '-j2']' returned non-zero exit status 2.

[erlend-aasland]

cc @erlend-aasland: Another build fix ;-)

Seems this fix broke some build bots ;)

[vstinner]

Seems this fix broke some build bots ;)

Nah, it didn't break buildbots, it just made a bug more visible :-) My following change is the real fix for test_freeze: PR #110451. It runs make distclean instead of make clean. I'm not sure why make clean doesn't remove python which was created by make! Maybe it's just more convenient like that.

[vstinner]

Nah, it didn't break buildbots, it just made a bug more visible :-) My following change is the real fix for test_freeze: PR #110451.

This following change fixed all buildbots. If I knew that it would be break something, I would do the two changes in the opposite order.

[erlend-aasland]

Great! Thanks :)