gh-106368: Add tests for permutation helpers in Argument Clinic

[erlend-aasland]

Added new test class PermutationTests()

• Issue: Increase Argument Clinic test coverage #106368

[sobolevn]

Looks good to me! 👍

[miss-islington]

Thanks @erlend-aasland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

[bedevere-bot]

GH-106409 is a backport of this pull request to the 3.12 branch.

[bedevere-bot]

GH-106410 is a backport of this pull request to the 3.11 branch.

[erlend-aasland]

Hm, I should have put these in the existing ClinicGroupPermuterTest test case. Let's keep that in mind and merge those two later, when we get the chance.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Arch Linux TraceRefs 3.x has failed when building commit 8f6df5e.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/484/builds/3602) and take a look at the build logs.
4. Check if the failure is related to this commit (8f6df5e) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/484/builds/3602

Failed tests:

• test__xxsubinterpreters

Summary of the results of the build (if available):

==

Click to see traceback logs

remote: Enumerating objects: 34, done. remote: Counting objects: 2% (1/34) remote: Counting objects: 5% (2/34) remote: Counting objects: 8% (3/34) remote: Counting objects: 11% (4/34) remote: Counting objects: 14% (5/34) remote: Counting objects: 17% (6/34) remote: Counting objects: 20% (7/34) remote: Counting objects: 23% (8/34) remote: Counting objects: 26% (9/34) remote: Counting objects: 29% (10/34) remote: Counting objects: 32% (11/34) remote: Counting objects: 35% (12/34) remote: Counting objects: 38% (13/34) remote: Counting objects: 41% (14/34) remote: Counting objects: 44% (15/34) remote: Counting objects: 47% (16/34) remote: Counting objects: 50% (17/34) remote: Counting objects: 52% (18/34) remote: Counting objects: 55% (19/34) remote: Counting objects: 58% (20/34) remote: Counting objects: 61% (21/34) remote: Counting objects: 64% (22/34) remote: Counting objects: 67% (23/34) remote: Counting objects: 70% (24/34) remote: Counting objects: 73% (25/34) remote: Counting objects: 76% (26/34) remote: Counting objects: 79% (27/34) remote: Counting objects: 82% (28/34) remote: Counting objects: 85% (29/34) remote: Counting objects: 88% (30/34) remote: Counting objects: 91% (31/34) remote: Counting objects: 94% (32/34) remote: Counting objects: 97% (33/34) remote: Counting objects: 100% (34/34) remote: Counting objects: 100% (34/34), done. remote: Compressing objects: 5% (1/17) remote: Compressing objects: 11% (2/17) remote: Compressing objects: 17% (3/17) remote: Compressing objects: 23% (4/17) remote: Compressing objects: 29% (5/17) remote: Compressing objects: 35% (6/17) remote: Compressing objects: 41% (7/17) remote: Compressing objects: 47% (8/17) remote: Compressing objects: 52% (9/17) remote: Compressing objects: 58% (10/17) remote: Compressing objects: 64% (11/17) remote: Compressing objects: 70% (12/17) remote: Compressing objects: 76% (13/17) remote: Compressing objects: 82% (14/17) remote: Compressing objects: 88% (15/17) remote: Compressing objects: 94% (16/17) remote: Compressing objects: 100% (17/17) remote: Compressing objects: 100% (17/17), done. remote: Total 20 (delta 16), reused 6 (delta 3), pack-reused 0  From https://github.com/python/cpython * branch main -> FETCH_HEAD Note: switching to '8f6df5e9cbc3a1689601714192aa6ecbb23e1927'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c <new-branch-name> Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 8f6df5e9cb gh-106368: Add tests for permutation helpers in Argument Clinic (#106407) Switched to and reset branch 'main' Objects/object.c:2197: _Py_ForgetReference: Assertion failed: invalid object chain Enable tracemalloc to get the memory block allocation traceback object address : 0x7fc2dd670ee0 object refcount : 0 object type : 0x55b810834bc0 object type name: str object repr : <refcnt 0 at 0x7fc2dd670ee0> Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed Python runtime state: initialized Current thread 0x00007fc2ef2de740 (most recent call first): <no Python frame> Debug memory block at address p=0x7fc2ef205e00: API '�' 18302063728033390045bytes originally requested The 7 pad bytes at p-7 are notall FORBIDDENBYTE (0xfd): at p-7: 0xdd***OUCH at p-6: 0xdd***OUCH at p-5: 0xdd***OUCH at p-4: 0xdd***OUCH at p-3: 0xdd***OUCH at p-2: 0xdd***OUCH at p-1: 0xdd***OUCH Because memory is corrupted at the start, the count of bytes requested may be bogus, and checking the trailing pad bytes may segfault. The 8 pad bytes at tail=0xfdfe7dc0ed1e3bdd are Fatal Python error: Segmentation fault Current thread 0x00007fc2ef2de740 (most recent call first): <no Python frame> Extension modules: _testcapi, _xxsubinterpreters (total: 2) make: *** [Makefile:2010: buildbottest] Segmentation fault (core dumped) Cannot open file '/buildbot/buildarea/3.x.pablogsal-arch-x86_64/build/test-results.xml' for upload

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL7 LTO 3.11 has failed when building commit 9f8aa7a.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/924/builds/913) and take a look at the build logs.
4. Check if the failure is related to this commit (9f8aa7a) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/924/builds/913

Failed tests:

• test___all__
• test_tools

Failed subtests:

• test_freeze_simple_script - test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script
• test_all - test.test___all__.AllTest.test_all

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

413 tests OK.

10 slowest tests:

• test_tools: 3 min 39 sec
• test_concurrent_futures: 2 min 27 sec
• test_multiprocessing_spawn: 2 min 14 sec
• test_multiprocessing_forkserver: 1 min 24 sec
• test_multiprocessing_fork: 1 min 14 sec
• test_tokenize: 1 min 9 sec
• test_asyncio: 1 min 4 sec
• test_peg_generator: 49.5 sec
• test_signal: 48.1 sec
• test_unparse: 41.5 sec

1 test failed:
test_tools

20 tests skipped:
test_devpoll test_gdb test_idle test_ioctl test_kqueue
test_launcher test_msilib test_smtpnet test_ssl test_startfile
test_tcl test_tix test_tk test_ttk_guionly test_ttk_textonly
test_turtle test_winconsoleio test_winreg test_winsound
test_zipfile64

2 re-run tests:
test___all__ test_tools

Total duration: 9 min 58 sec

Click to see traceback logs

Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test_tools/test_freeze.py", line 28, in test_freeze_simple_script executable = helper.freeze(python, scriptfile, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 198, in freeze _run_quiet([MAKE, '-C', os.path.dirname(scriptfile)]) File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 25, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmpuzyw3yoh']' returned non-zero exit status 2. Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/multiprocessing/resource_tracker.py", line 209, in main cache[rtype].remove(name) KeyError: '/psm_79b4645b' Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/tmpuzyw3yoh/cpython/Lib/sysconfig.py", line 715, in get_config_varreturn get_config_vars().get(name) ^^^^^^^^^^^^^^^^^ File "/tmp/tmpuzyw3yoh/cpython/Lib/sysconfig.py", line 670, in get_config_vars _init_posix(_CONFIG_VARS) File "/tmp/tmpuzyw3yoh/cpython/Lib/sysconfig.py", line 531, in _init_posix _temp =__import__(name, globals(), locals(), ['build_time_vars'], 0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ModuleNotFoundError: No module named '_sysconfigdata__linux_s390x-linux-gnu' Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test_tools/test_freeze.py", line 28, in test_freeze_simple_script executable = helper.freeze(python, scriptfile, outdir) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 198, in freeze _run_quiet([MAKE, '-C', os.path.dirname(scriptfile)]) File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Tools/freeze/test/freeze.py", line 25, in _run_quietreturn subprocess.run( ^^^^^^^^^^^^^^^ File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/subprocess.py", line 571, in runraise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/make', '-C', '/tmp/tmp4po6o4wf']' returned non-zero exit status 2. Traceback (most recent call last): File "<string>", line 1, in <module> File "/tmp/tmp4po6o4wf/cpython/Lib/sysconfig.py", line 715, in get_config_varreturn get_config_vars().get(name) ^^^^^^^^^^^^^^^^^ File "/tmp/tmp4po6o4wf/cpython/Lib/sysconfig.py", line 670, in get_config_vars _init_posix(_CONFIG_VARS) File "/tmp/tmp4po6o4wf/cpython/Lib/sysconfig.py", line 531, in _init_posix _temp =__import__(name, globals(), locals(), ['build_time_vars'], 0) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ModuleNotFoundError: No module named '_sysconfigdata__linux_s390x-linux-gnu' Traceback (most recent call last): File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test___all__.py", line 129, in test_allself.check_all(modname) File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test___all__.py", line 43, in check_allwith warnings_helper.check_warnings( File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/contextlib.py", line 144, in __exit__next(self.gen) File "/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/support/warnings_helper.py", line 185, in _filterwarningsraiseAssertionError("unhandled warning %s"% reraise[0]) AssertionError: unhandled warning{message : DeprecationWarning("invalid escape sequence '\\?'"), category : 'DeprecationWarning', filename : '/home/dje/cpython-buildarea/3.11.edelsohn-rhel-z.lto/build/Lib/test/test_httpservers.py', lineno : 445, line : None}