gh-104909: Split some more insts into ops

[gvanrossum]

I started by adding some code that shows the most deserving non-viable opcodes. Also doubling max trace size again.

I could do this all day, but the stats show there are diminishing returns. Output from python3 Tools/cases_generator/generate_cases.py -v before splitting the top 5:

Read 186 instructions, 46 ops, 207 macros, 12 pseudos, and 14 families from Python/bytecodes.c The following ops are not viable uops: 1,906,418,185 LOAD_ATTR_METHOD_WITH_VALUES (specialization of LOAD_ATTR) 1,408,928,326 LOAD_ATTR_METHOD_NO_DICT (specialization of LOAD_ATTR) 1,359,753,529 LOAD_ATTR_SLOT (specialization of LOAD_ATTR) 1,064,113,118 STORE_ATTR_SLOT (specialization of STORE_ATTR) 981,677,187 STORE_ATTR_INSTANCE_VALUE (specialization of STORE_ATTR) 958,283,266 YIELD_VALUE 777,829,833 CALL (unspecialized) 490,635,610 SEND_GEN (specialization of SEND) 436,706,438 JUMP_FORWARD 430,588,960 EXTENDED_ARG 391,017,977 JUMP_BACKWARD_NO_INTERRUPT 351,940,012 LOAD_ATTR_MODULE (specialization of LOAD_ATTR) 347,159,058 LOAD_ATTR_WITH_HINT (specialization of LOAD_ATTR) 308,391,927 FOR_ITER (unspecialized) 251,204,370 RETURN_GENERATOR 241,313,761 CALL_LIST_APPEND (specialization of CALL) 177,791,632 CALL_PY_WITH_DEFAULTS (specialization of CALL) 175,204,354 CALL_KW 163,262,604 FOR_ITER_GEN (specialization of FOR_ITER) 146,454,299 BINARY_SUBSCR_GETITEM (specialization of BINARY_SUBSCR) 143,423,491 LOAD_ATTR_NONDESCRIPTOR_WITH_VALUES (specialization of LOAD_ATTR) 136,661,968 STORE_FAST_LOAD_FAST 136,511,856 LOAD_ATTR_CLASS (specialization of LOAD_ATTR) 112,730,831 SEND (unspecialized) 92,715,546 MAKE_CELL 83,410,408 CALL_FUNCTION_EX 69,188,245 CALL_ALLOC_AND_ENTER_INIT (specialization of CALL) 56,944,041 LOAD_ATTR_METHOD_LAZY_DICT (specialization of LOAD_ATTR) 52,280,673 LOAD_ATTR_PROPERTY (specialization of LOAD_ATTR) 50,964,787 STORE_ATTR_WITH_HINT (specialization of STORE_ATTR) 22,352,640 LOAD_ATTR_NONDESCRIPTOR_NO_DICT (specialization of LOAD_ATTR) 8,497,626 RERAISE 6,000,000 END_ASYNC_FOR 5,253,976 BEFORE_WITH 2,897,006 RAISE_VARARGS 2,005,368 IMPORT_FROM 1,968,468 IMPORT_NAME 8,160 BEFORE_ASYNC_WITH 5,907 RESUME (unspecialized) 1,540 LOAD_SUPER_ATTR (unspecialized) 801 CLEANUP_THROW ENTER_EXECUTOR LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN (specialization of LOAD_ATTR) 

I took the counts from here (follow the pystats raw link).

• Curious what's up with LOAD_ATTR_GETATTRIBUTE_OVERRIDDEN? It was the same on the weekly run a week before; the week before that (Sept 10) the file format was different. @brandtbucher any intuition on this?

• Issue: Split opcodes into micro-ops #104909

[gvanrossum]

Another couple of questions for @brandtbucher (or @markshannon).

• Sometimes there's duplication between two uops, e.g. _GUARD_DORV_VALUES and _STORE_ATTR_INSTANCE_VALUE both set dorv = *_PyObject_DictOrValuesPointer(owner);. In Tier 1 the assumption has always been that the C compiler's optimizer will eliminate the duplication (it's a static inline function). But in the Tier 2 interpreter, we end up executing that code twice (once in each uop), and in the copy-and-patch machine code it also takes up twice the space. It doesn't look too bad, because after removing asserts it's just obj - 3, and a modern CPU should do this very fast, but there might be other occurrences of this kind of duplication that are slightly less minimal.
• There's an assert(type_version != 0); in the _GUARD_TYPE_VERSION macro. This existed in most of the instructions I split up, except in LOAD_ATTR_METHOD_NO_DICT. Was that an oversight, or can it be zero for that opcode? (If so, then my PR is incorrect.)
• In _LOAD_ATTR_SLOT, which is an "action" uop, I left a DEOPT_IF call. This is unprincipled, but it seems to work, and splitting this off into its own guard would duplicate much of the work this uop does (in particular the indirect load). Thoughts?

[Fidget-Spinner]

• In _LOAD_ATTR_SLOT, which is an "action" uop, I left a DEOPT_IF call. This is unprincipled, but it seems to work, and splitting this off into its own guard would duplicate much of the work this uop does (in particular the indirect load). Thoughts?

Not Mark or Brandt, but on the optimization side, we would have to treat this as an impure op, which makes most of the optimization opportunities go away. However, we're likely converting some of these to LOAD_CONST anyways so that might mitigate it?

[gvanrossum]

Not Mark or Brandt, but on the optimization side, we would have to treat this as an impure op, which makes most of the optimization opportunities go away.

We already have other actions using DEOPT_IF (e.g. _LOAD_GLOBAL_BUILTINS). We can split all these up later. My first goal is just to be able to translate more bytecodes to Tier 2, I'm slowly working my way through the most frequently executed instructions (per the table shown above).

However, we're likely converting some of these to LOAD_CONST anyways so that might mitigate it?

That's still a bit in the future though. And I'm not sure how often LOAD_ATTR_SLOT in particular would become LOAD_CONST (since it specifically fetches an instance attribute). It seems more likely for some other variants like LOAD_ATTR_MODULE, and that would probably load the result from the cache, not be transformed in (some guard +) LOAD_CONST.

[gvanrossum]

Going to merge this soon, basically as-is. Perf is same as base (1.00 faster).

[gvanrossum]

I can't find a persistent pattern in the failing tests. Maybe it's a timeout in multiprocessing? I believe @vstinner recently made flaky tests more likely to cause CI failure, maybe it's that?

[gvanrossum]

Well, okay, some tests fail consistently, so I have to bisect. Sorry for the accusations.

[gvanrossum]

Looks like LOAD_ATTR_METHOD_WITH_VALUES is the culprit.

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Fedora Stable LTO + PGO 3.x has failed when building commit 5bb6f0f.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/29/builds/4838) and take a look at the build logs.
4. Check if the failure is related to this commit (5bb6f0f) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/29/builds/4838

Failed tests:

• test.test_concurrent_futures.test_shutdown

Failed subtests:

• test_interpreter_shutdown - test.test_concurrent_futures.test_shutdown.ProcessPoolForkserverProcessPoolShutdownTest.test_interpreter_shutdown

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/test/test_concurrent_futures/test_shutdown.py", line 49, in test_interpreter_shutdownself.assertFalse(err) AssertionError: b'Exception in thread Thread-1:\nTraceback (most recent call last):\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/threading.py", line 1066, in _bootstrap_inner\n self.run()\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/concurrent/futures/process.py", line 344, in run\n self.add_call_item_to_queue()\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/concurrent/futures/process.py", line 399, in add_call_item_to_queue\n self.call_queue.put(_CallItem(work_id,\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/queues.py", line 94, in put\n self._start_thread()\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/queues.py", line 177, in _start_thread\n self._thread.start()\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/threading.py", line 985, in start\n _start_new_thread(self._bootstrap, ())\nRuntimeError: can\'t create new thread at interpreter shutdown\nTraceback (most recent call last):\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/forkserver.py", line 274, in main\n code = _serve_one(child_r, fds,\n ^^^^^^^^^^^^^^^^^^^^^^^^\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/forkserver.py", line 313, in _serve_one\n code = spawn._main(child_r, parent_sentinel)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/spawn.py", line 132, in _main\n self = reduction.pickle.load(from_parent)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File "/home/buildbot/buildarea/3.x.cstratak-fedora-stable-x86_64.lto-pgo/build/Lib/multiprocessing/synchronize.py", line 115, in __setstate__\n self._semlock = _multiprocessing.SemLock._rebuild(*state)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nFileNotFoundError: [Errno 2] No such file or directory\n' is not false 

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian PGO 3.x has failed when building commit 5bb6f0f.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/249/builds/6464) and take a look at the build logs.
4. Check if the failure is related to this commit (5bb6f0f) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/249/builds/6464

Failed tests:

• test.test_multiprocessing_fork.test_processes

Summary of the results of the build (if available):

==

Click to see traceback logs

remote: Enumerating objects: 29, done. remote: Counting objects: 3% (1/29) remote: Counting objects: 6% (2/29) remote: Counting objects: 10% (3/29) remote: Counting objects: 13% (4/29) remote: Counting objects: 17% (5/29) remote: Counting objects: 20% (6/29) remote: Counting objects: 24% (7/29) remote: Counting objects: 27% (8/29) remote: Counting objects: 31% (9/29) remote: Counting objects: 34% (10/29) remote: Counting objects: 37% (11/29) remote: Counting objects: 41% (12/29) remote: Counting objects: 44% (13/29) remote: Counting objects: 48% (14/29) remote: Counting objects: 51% (15/29) remote: Counting objects: 55% (16/29) remote: Counting objects: 58% (17/29) remote: Counting objects: 62% (18/29) remote: Counting objects: 65% (19/29) remote: Counting objects: 68% (20/29) remote: Counting objects: 72% (21/29) remote: Counting objects: 75% (22/29) remote: Counting objects: 79% (23/29) remote: Counting objects: 82% (24/29) remote: Counting objects: 86% (25/29) remote: Counting objects: 89% (26/29) remote: Counting objects: 93% (27/29) remote: Counting objects: 96% (28/29) remote: Counting objects: 100% (29/29) remote: Counting objects: 100% (29/29), done. remote: Compressing objects: 6% (1/15) remote: Compressing objects: 13% (2/15) remote: Compressing objects: 20% (3/15) remote: Compressing objects: 26% (4/15) remote: Compressing objects: 33% (5/15) remote: Compressing objects: 40% (6/15) remote: Compressing objects: 46% (7/15) remote: Compressing objects: 53% (8/15) remote: Compressing objects: 60% (9/15) remote: Compressing objects: 66% (10/15) remote: Compressing objects: 73% (11/15) remote: Compressing objects: 80% (12/15) remote: Compressing objects: 86% (13/15) remote: Compressing objects: 93% (14/15) remote: Compressing objects: 100% (15/15) remote: Compressing objects: 100% (15/15), done. remote: Total 15 (delta 14), reused 1 (delta 0), pack-reused 0  From https://github.com/python/cpython * branch main -> FETCH_HEAD Note: switching to '5bb6f0fcba663e1006f9063d1027ce8bd9f8effb'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c <new-branch-name> Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 5bb6f0fcba gh-104909: Split some more insts into ops (#109943) Switched to and reset branch 'main' find: ‘build’: No such file or directoryfind: ‘build’: No such file or directoryfind: ‘build’: No such file or directoryfind: ‘build’: No such file or directorymake[2]: [Makefile:2821: clean-retain-profile] Error 1 (ignored) In file included from Python/ceval.c:766: Python/generated_cases.c.h: In function ‘_PyEval_EvalFrameDefault’: Python/generated_cases.c.h:2634:31: warning: unused variable ‘tp’ [-Wunused-variable] 2634 | PyTypeObject *tp = Py_TYPE(owner); |^~ Python/generated_cases.c.h:3593:31: warning: unused variable ‘owner_cls’ [-Wunused-variable] 3593 | PyTypeObject *owner_cls = Py_TYPE(owner); |^~~~~~~~~ Python/generated_cases.c.h:3642:31: warning: unused variable ‘owner_cls’ [-Wunused-variable] 3642 | PyTypeObject *owner_cls = Py_TYPE(owner); |^~~~~~~~~ In file included from Python/executor.c:87: Python/executor_cases.c.h: In function ‘_PyUopExecute’: Python/executor_cases.c.h:1739:27: warning: unused variable ‘tp’ [-Wunused-variable] 1739 | PyTypeObject *tp = Py_TYPE(owner); |^~ Python/executor_cases.c.h:2302:27: warning: unused variable ‘owner_cls’ [-Wunused-variable] 2302 | PyTypeObject *owner_cls = Py_TYPE(owner); |^~~~~~~~~ Python/executor_cases.c.h:2348:27: warning: unused variable ‘owner_cls’ [-Wunused-variable] 2348 | PyTypeObject *owner_cls = Py_TYPE(owner); |^~~~~~~~~./Modules/readline.c: In function ‘setup_readline’: ./Modules/readline.c:1256:21: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types] 1256 | rl_startup_hook = on_startup_hook; |^ ./Modules/readline.c:1258:23: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types] 1258 | rl_pre_input_hook = on_pre_input_hook; |^./Modules/socketmodule.c: In function ‘getsockaddrarg’: ./Modules/socketmodule.c:2468:9: warning: ‘strncpy’ specified bound 64 equals destination size [-Wstringop-truncation] 2468 | strncpy((char *)sa->salg_name, name, sizeof(sa->salg_name)); |^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./Modules/_decimal/libmpdec/context.c:56: warning: mpd_setminalloc: ignoring request to set MPD_MINALLOC a second time ./Modules/_decimal/libmpdec/context.c:56: warning: mpd_setminalloc: ignoring request to set MPD_MINALLOC a second time In file included from Python/ceval.c:766: Python/generated_cases.c.h: In function ‘_PyEval_EvalFrameDefault’: Python/generated_cases.c.h:2634:31: warning: unused variable ‘tp’ [-Wunused-variable] 2634 | PyTypeObject *tp = Py_TYPE(owner); |^~ Python/generated_cases.c.h:3593:31: warning: unused variable ‘owner_cls’ [-Wunused-variable] 3593 | PyTypeObject *owner_cls = Py_TYPE(owner); |^~~~~~~~~ Python/generated_cases.c.h:3642:31: warning: unused variable ‘owner_cls’ [-Wunused-variable] 3642 | PyTypeObject *owner_cls = Py_TYPE(owner); |^~~~~~~~~ In file included from Python/executor.c:87: Python/executor_cases.c.h: In function ‘_PyUopExecute’: Python/executor_cases.c.h:1739:27: warning: unused variable ‘tp’ [-Wunused-variable] 1739 | PyTypeObject *tp = Py_TYPE(owner); |^~ Python/executor_cases.c.h:2302:27: warning: unused variable ‘owner_cls’ [-Wunused-variable] 2302 | PyTypeObject *owner_cls = Py_TYPE(owner); |^~~~~~~~~ Python/executor_cases.c.h:2348:27: warning: unused variable ‘owner_cls’ [-Wunused-variable] 2348 | PyTypeObject *owner_cls = Py_TYPE(owner); |^~~~~~~~~./Modules/readline.c: In function ‘setup_readline’: ./Modules/readline.c:1256:21: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types] 1256 | rl_startup_hook = on_startup_hook; |^ ./Modules/readline.c:1258:23: warning: assignment to ‘int (*)(const char *, int)’ from incompatible pointer type ‘int (*)(void)’ [-Wincompatible-pointer-types] 1258 | rl_pre_input_hook = on_pre_input_hook; |^ In function ‘utf8_toUtf8’, inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3, inlined from ‘doParseXmlDecl’ at ./Modules/expat/xmltok.c:1197:13: ./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=] 390 | memcpy(*toP, *fromP, bytesToCopy); |^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In function ‘utf8_toUtf8’, inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3, inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1075:9: ./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=] 390 | memcpy(*toP, *fromP, bytesToCopy); |^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In function ‘utf8_toUtf8’, inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3, inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1081:12: ./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=] 390 | memcpy(*toP, *fromP, bytesToCopy); |^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In function ‘utf8_toUtf8’, inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3, inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1088:9: ./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=] 390 | memcpy(*toP, *fromP, bytesToCopy); |^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=] In function ‘utf8_toUtf8’, inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3, inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1115:7: ./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=] 390 | memcpy(*toP, *fromP, bytesToCopy); |^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In function ‘utf8_toUtf8’, inlined from ‘toAscii’ at ./Modules/expat/xmltok.c:1043:3, inlined from ‘parsePseudoAttribute.part.0’ at ./Modules/expat/xmltok.c:1128:9: ./Modules/expat/xmltok.c:390:5: warning: ‘memcpy’ writing 2 or more bytes into a region of size 1 overflows the destination [-Wstringop-overflow=] 390 | memcpy(*toP, *fromP, bytesToCopy); |^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~make: *** [Makefile:2040: buildbottest] Error 5

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot s390x RHEL8 LTO 3.x has failed when building commit 5bb6f0f.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/567/builds/4991) and take a look at the build logs.
4. Check if the failure is related to this commit (5bb6f0f) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/567/builds/4991

Failed tests:

• test.test_multiprocessing_fork.test_processes

Summary of the results of the build (if available):

==

Click to see traceback logs

Note: switching to '5bb6f0fcba663e1006f9063d1027ce8bd9f8effb'. You are in 'detached HEAD' state. You can look around, make experimental changes and commit them, and you can discard any commits you make in this state without impacting any branches by switching back to a branch. If you want to create a new branch to retain commits you create, you may do so (now or later) by using -c with the switch command. Example: git switch -c <new-branch-name> Or undo this operation with: git switch - Turn off this advice by setting config variable advice.detachedHead to false HEAD is now at 5bb6f0fcba gh-104909: Split some more insts into ops (#109943) Switched to and reset branch 'main' In file included from Python/ceval.c:766: Python/generated_cases.c.h: In function ‘_PyEval_EvalFrameDefault’: Python/generated_cases.c.h:2634:31: warning: unused variable ‘tp’ [-Wunused-variable] PyTypeObject *tp = Py_TYPE(owner);^~ Python/generated_cases.c.h:3593:31: warning: unused variable ‘owner_cls’ [-Wunused-variable] PyTypeObject *owner_cls = Py_TYPE(owner);^~~~~~~~~ Python/generated_cases.c.h:3642:31: warning: unused variable ‘owner_cls’ [-Wunused-variable] PyTypeObject *owner_cls = Py_TYPE(owner);^~~~~~~~~ In file included from Python/executor.c:87: Python/executor_cases.c.h: In function ‘_PyUopExecute’: Python/executor_cases.c.h:1739:27: warning: unused variable ‘tp’ [-Wunused-variable] PyTypeObject *tp = Py_TYPE(owner);^~ Python/executor_cases.c.h:2302:27: warning: unused variable ‘owner_cls’ [-Wunused-variable] PyTypeObject *owner_cls = Py_TYPE(owner);^~~~~~~~~ Python/executor_cases.c.h:2348:27: warning: unused variable ‘owner_cls’ [-Wunused-variable] PyTypeObject *owner_cls = Py_TYPE(owner);^~~~~~~~~ ./Include/internal/pycore_semaphore.h:54:1: warning: type of ‘_PySemaphore_Wakeup’ does not match original declaration [-Wlto-type-mismatch] _PySemaphore_Wakeup(_PySemaphore *sema); ^ Python/parking_lot.c:197:1: note: ‘_PySemaphore_Wakeup’ was previously declared here _PySemaphore_Wakeup(_PySemaphore *sema) ^ Python/parking_lot.c:197:1: note: code may be misoptimized unless -fno-strict-aliasing is used ./Include/internal/pycore_semaphore.h:57:18: warning: type of ‘_PySemaphore_Init’ does not match original declaration [-Wlto-type-mismatch] PyAPI_FUNC(void) _PySemaphore_Init(_PySemaphore *sema); ^ Python/parking_lot.c:53:1: note: ‘_PySemaphore_Init’ was previously declared here _PySemaphore_Init(_PySemaphore *sema) ^ Python/parking_lot.c:53:1: note: code may be misoptimized unless -fno-strict-aliasing is used ./Include/internal/pycore_semaphore.h:49:1: warning: type of ‘_PySemaphore_Wait’ does not match original declaration [-Wlto-type-mismatch] _PySemaphore_Wait(_PySemaphore *sema, _PyTime_t timeout_ns, int detach); ^ Python/parking_lot.c:178:1: note: ‘_PySemaphore_Wait’ was previously declared here _PySemaphore_Wait(_PySemaphore *sema, _PyTime_t timeout, int detach) ^ Python/parking_lot.c:178:1: note: code may be misoptimized unless -fno-strict-aliasing is used ./Include/internal/pycore_semaphore.h:58:18: warning: type of ‘_PySemaphore_Destroy’ does not match original declaration [-Wlto-type-mismatch] PyAPI_FUNC(void) _PySemaphore_Destroy(_PySemaphore *sema); ^ Python/parking_lot.c:81:1: note: ‘_PySemaphore_Destroy’ was previously declared here _PySemaphore_Destroy(_PySemaphore *sema) ^ Python/parking_lot.c:81:1: note: code may be misoptimized unless -fno-strict-aliasing is used make: *** [Makefile:2040: buildbottest] Error 5

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot ARM Raspbian 3.x has failed when building commit 5bb6f0f.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/424/builds/5033) and take a look at the build logs.
4. Check if the failure is related to this commit (5bb6f0f) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/424/builds/5033

Failed tests:

• test.test_multiprocessing_spawn.test_misc

Failed subtests:

• test_dict - test.test_multiprocessing_spawn.test_misc.TestSyncManagerTypes.test_dict
• test_bounded_semaphore - test.test_multiprocessing_spawn.test_misc.TestSyncManagerTypes.test_bounded_semaphore
• test_condition - test.test_multiprocessing_spawn.test_misc.TestSyncManagerTypes.test_condition

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last): File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5931, in test_dictself.run_worker(self._test_dict, o) File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5799, in run_workerself.wait_proc_exit() File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5781, in wait_proc_exit join_process(self.proc) File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 100, in join_process threading_helper.join_thread(process) File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/threading_helper.py", line 113, in join_threadraiseAssertionError(msg) AssertionError: failed to join the thread in 52.5 seconds Traceback (most recent call last): File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5854, in test_conditionself.run_worker(self._test_condition, o) File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5799, in run_workerself.wait_proc_exit() File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5781, in wait_proc_exit join_process(self.proc) File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 100, in join_process threading_helper.join_thread(process) File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/threading_helper.py", line 113, in join_threadraiseAssertionError(msg) AssertionError: failed to join the thread in 52.5 seconds Traceback (most recent call last): File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5845, in test_bounded_semaphoreself.test_semaphore(sname="BoundedSemaphore") File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5841, in test_semaphoreself.run_worker(self._test_semaphore, o) File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5799, in run_workerself.wait_proc_exit() File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 5781, in wait_proc_exit join_process(self.proc) File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/_test_multiprocessing.py", line 100, in join_process threading_helper.join_thread(process) File "/var/lib/buildbot/workers/3.x.gps-raspbian.nondebug/build/Lib/test/support/threading_helper.py", line 113, in join_threadraiseAssertionError(msg) AssertionError: failed to join the thread in 52.5 seconds

[vstinner]

I believe @vstinner recently made flaky tests more likely to cause CI failure, maybe it's that?

Right. Tests are now run with --rerun-fail on GitHub Actions and buildbots: any test marks the whole build as a failure.