test_math.test_sumprod_stress() takes 5 minutes on debug build

[vstinner]

On the " PPC64LE Fedora Stable Clang 3.x" with Python build by "/configure CC=clang LD=clang --with-pydebug", test_math takes 7 min 39 sec:

make buildbottest 'TESTOPTS=-j2 --junit-xml test-results.xml -j10 --fail-rerun ${BUILDBOT_TESTOPTS}' TESTPYTHONOPTS= TESTTIMEOUT=1200
== CPython 3.13.0a0 (heads/main:269005e784, Oct 3 2023, 23:52:56) [Clang 16.0.6 (Fedora 16.0.6-3.fc38)]
== Linux-5.8.17-200.fc32.ppc64le-ppc64le-with-glibc2.37 little-endian
== Python build: debug
== cwd: /home/buildbot/buildarea/3.x.cstratak-fedora-stable-ppc64le.clang/build/build/test_python_worker_2081694æ
== CPU count: 8
== encodings: locale=UTF-8, FS=utf-8
== resources (10): audio, cpu, curses, decimal, gui, largefile, network, subprocess, urlfetch, walltime

(...)
10 slowest tests:
- test_math: 7 min 39 sec
- test.test_multiprocessing_spawn.test_processes: 2 min 49 sec
- test_pathlib: 2 min 36 sec
- test_unparse: 2 min 30 sec
- test_tokenize: 2 min 29 sec
- test.test_multiprocessing_forkserver.test_processes: 2 min 22 sec
- test_capi: 1 min 52 sec
- test_cppext: 1 min 42 sec
- test_unicodedata: 1 min 39 sec
- test_tarfile: 1 min 33 sec

Can we split test_math into a test package made of multiple tests to run them in parallel?

Should we remove/skip some slow tests? Buildbot uses --slow-ci which enables -u all resources.

[vstinner]

Timings as JUnit XML of ./python -m test test_math -v --junit-xml=results-all.xml -u all: results-all.xml.txt (5 min 28 sec).

[vstinner]

Timings as JUnit XML of ./python -m test test_math -v --junit-xml=results-all.xml -u all: results-all.xml.txt (5 min 28 sec).

test_math.test_sumprod_stress() alone takes 5 min 13 sec. The test was added by issue GH-100485: commit 47b9f83.

[vstinner]

test_math.test_sumprod_stress() alone takes 5 min 13 sec

This test uses 9 pools of lengths: [10, 9, 9, 11, 10, 9, 8, 5, 8]. It uses 4 nested loops for a total of 5,969,664 iterations.

IMO that's excessive and it should be possible to minimize the test to a few cases. I don't think that there is a need to test 5,969,664 cases.

[vstinner]

@rhettinger @serhiy-storchaka: Can you please have a look at test_math.test_sumprod_stress() and see if we can test a few cases, rather than testing 5,969,664 cases?

On my laptop with Python built with gcc -O0, ./python -m test test_math -u all takes 10 min 50 sec :-(

[vstinner]

• On Python built with gcc -O3 (./configure), ./python -m test test_math -u all -v -m test_sumprod_stress takes 25 seconds.
• On Python built with gcc -Og (./configure --with-pydebug), ./python -m test test_math -u all -v -m test_sumprod_stress takes 1 min 49 sec.

[serhiy-storchaka]

Sorry, I don't know exactly what it is testing and what can be cut out without losing the value of the test.

[skirpichev]

if we can test a few cases, rather than testing 5,969,664 cases?

@vstinner, test_sumprod_stress() should cover some lines, mentioned in #110000 (e.g. overflows for big integers), but I think at least in this function we could test only "exact" floats/ints, while mixed inputs could go to the testSumProd().

[vstinner]

Well, apparently, other people don't care much, so let's leave it like that for now. If someone wants to propose a change, please open a new issue.