Skip to content

Conversation

@serhiy-storchaka
Copy link
Member

@serhiy-storchakaserhiy-storchaka commented Aug 25, 2023

@serhiy-storchaka
Copy link
MemberAuthor

These are mostly tests which contain several sleep()s or call a function which expected to fail by timeout. There are few tests which depend on slow external network service.

@serhiy-storchaka
Copy link
MemberAuthor

$ ./python -m test -uall,-cpu -j4 ... Total duration: 7 min 27 sec Total tests: run=42,127 skipped=1,077 
$ ./python -m test -uall,-cpu,-walltime -j4 ... Total duration: 6 min 26 sec Total tests: run=42,127 skipped=1,139 

@miss-islington
Copy link
Contributor

Thanks @serhiy-storchaka for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11, 3.12.
🐍🍒⛏🤖

@serhiy-storchakaserhiy-storchaka deleted the mark-slow-tests-walltime branch September 5, 2023 14:56
@miss-islington
Copy link
Contributor

Sorry, @serhiy-storchaka, I could not cleanly backport this to 3.11 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 1e0d62793a84001e92f1c80b511d3a212b435acc 3.11

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Sep 5, 2023
…es_resource('walltime') (pythonGH-108480) (cherry picked from commit 1e0d627) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-bot
Copy link

GH-108923 is a backport of this pull request to the 3.12 branch.

@bedevere-botbedevere-bot removed the needs backport to 3.12 only security fixes label Sep 5, 2023
serhiy-storchaka added a commit to serhiy-storchaka/cpython that referenced this pull request Sep 5, 2023
… requires_resource('walltime') (pythonGH-108480). (cherry picked from commit 1e0d627) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@bedevere-bot
Copy link

GH-108924 is a backport of this pull request to the 3.11 branch.

@bedevere-botbedevere-bot removed the needs backport to 3.11 only security fixes label Sep 5, 2023
serhiy-storchaka added a commit that referenced this pull request Sep 5, 2023
vstinner pushed a commit to vstinner/cpython that referenced this pull request Sep 5, 2023
Yhg1s pushed a commit that referenced this pull request Sep 8, 2023
* gh-108834: regrtest reruns failed tests in subprocesses (#108839) When using --rerun option, regrtest now re-runs failed tests in verbose mode in fresh worker processes to have more deterministic behavior. So it can write its final report even if a test killed a worker progress. Add --fail-rerun option to regrtest: exit with non-zero exit code if a test failed pass passed when re-run in verbose mode (in a fresh process). That's now more useful since tests can pass when re-run in a fresh worker progress, whereas they failed when run after other tests when tests are run sequentially. Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a deprecated alias. Changes: * Fix and enhance statistics in regrtest summary. Add "(filtered)" when --match and/or --ignore options are used. * Add RunTests class. * Add TestResult.get_rerun_match_tests() method * Rewrite code to serialize/deserialize worker arguments as JSON using a new WorkerJob class. * Fix stats when a test is run with --forever --rerun. * If failed test names cannot be parsed, log a warning and don't filter tests. * test_regrtest.test_rerun_success() now uses a marker file, since the test is re-run in a separated process. * Add tests on normalize_test_name() function. * Add test_success() and test_skip() tests to test_regrtest. (cherry picked from commit 31c2945) * gh-108834: regrtest --fail-rerun exits with code 5 (#108896) When the --fail-rerun option is used and a test fails and then pass, regrtest now uses exit code 5 ("rerun) instead of 2 ("bad test"). (cherry picked from commit 1170d5a) * gh-108416: Mark slow but not CPU bound test methods with requires_resource('walltime') (GH-108480) (cherry picked from commit 1e0d627) * Manually sync Lib/test/libregrtest/ from main --------- Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
@serhiy-storchakaserhiy-storchaka removed their assignment Sep 8, 2023
Sign up for freeto join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip newstestsTests in the Lib/test dir

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

@serhiy-storchaka@miss-islington@bedevere-bot@gpshead