gh-106368: Increase Argument Clinic CLI test coverage#107156

erlend-aasland · 2023-07-23T21:12:50Z

Instead of hacking into the Clinic class, use the Argument Clinic tool
to run the ~~ClinicWholeFileTest~~ ClinicExternalTest test suite.

TODO:

Remove tmp stuff from test_external, use -o to redirect the output, and simply compare the redirected output with clinic.test.c
A new test that verifies that the file is not touched if no changes are made (currently a part of test_external)
A test that verifies that an incorrect output checksum (and the generated output) is regenerated if -f is given
Test verbose mode

Issue: Increase Argument Clinic test coverage #106368

Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicWholeFileTest test suite.

test_external: run clinic on clinic.test.c test_no_change: check that dest is left unchanged if content does not change

erlend-aasland · 2023-07-23T22:38:18Z

FWIW; this messes up test coverage locally; that's solvable (IIRC), though.

erlend-aasland · 2023-07-23T22:41:00Z

The coveragerc file that's included in the sources disables coverage for if __name__ == "__main__", which is unfortunate; we should change that.

sobolevn

Better error messages :)

Lib/test/test_clinic.py

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

erlend-aasland · 2023-07-24T09:04:42Z

Thanks for the review, Nikita!

sobolevn

👍

Lib/test/test_clinic.py

sobolevn

Looks great! 👍

miss-islington · 2023-07-24T18:51:22Z

Thanks @erlend-aasland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.12.
🐍🍒⛏🤖

miss-islington · 2023-07-24T18:51:23Z

Thanks @erlend-aasland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.11.
🐍🍒⛏🤖

…-107156) Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicExternalTest test suite. (cherry picked from commit 83a2837) Co-authored-by: Erlend E. Aasland <erlend@python.org> Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-bot · 2023-07-24T18:51:33Z

GH-107189 is a backport of this pull request to the 3.12 branch.

…-107156) Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicExternalTest test suite. (cherry picked from commit 83a2837) Co-authored-by: Erlend E. Aasland <erlend@python.org> Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

bedevere-bot · 2023-07-24T18:51:37Z

GH-107190 is a backport of this pull request to the 3.11 branch.

…) (#107189) Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicExternalTest test suite. (cherry picked from commit 83a2837) Co-authored-by: Erlend E. Aasland <erlend@python.org> Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

…) (#107190) Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicExternalTest test suite. (cherry picked from commit 83a2837) Co-authored-by: Erlend E. Aasland <erlend@python.org> Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

…ythonGH-107156) (python#107190) Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicExternalTest test suite. (cherry picked from commit 83a2837) Co-authored-by: Erlend E. Aasland <erlend@python.org> Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

…07156) Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicExternalTest test suite. Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

pythongh-106368: Increase Argument Clinic CLI test coverage
12c0dc7
Instead of hacking into the Clinic class, use the Argument Clinic tool to run the ClinicWholeFileTest test suite.

bedevere-bot mentioned this pull request Jul 23, 2023
Increase Argument Clinic test coverage #106368
Closed

bedevere-bot added the tests Tests in the Lib/test dir label Jul 23, 2023

erlend-aasland added the skip news label Jul 23, 2023

erlend-aasland added 5 commits July 23, 2023 23:23

Add test for the --converters opt
e09e416

Fix converters test
f9d637b

Split test_external in two
9f617de
test_external: run clinic on clinic.test.c test_no_change: check that dest is left unchanged if content does not change

Test force/no-force runs
39ee0eb

Test verbose mode
a3f505e

erlend-aasland marked this pull request as ready for review July 23, 2023 22:36

erlend-aasland requested a review from AlexWaygood as a code owner July 23, 2023 22:36

bedevere-bot added the awaiting core review label Jul 23, 2023

erlend-aasland requested a review from sobolevn July 23, 2023 22:37

Add test cleanup to test_external
ff6a33c

sobolevn reviewed Jul 24, 2023
View reviewed changes

Lib/test/test_clinic.py Outdated Show resolvedHide resolved
Lib/test/test_clinic.py Outdated Show resolvedHide resolved

erlend-aaslandand others added 2 commits July 24, 2023 10:12

Update Lib/test/test_clinic.py
e97414c
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

Update Lib/test/test_clinic.py
e048663
Co-authored-by: Nikita Sobolev <mail@sobolevn.me>

sobolevn approved these changes Jul 24, 2023
View reviewed changes

erlend-aasland commented Jul 24, 2023
View reviewed changes

Lib/test/test_clinic.py Outdated Show resolvedHide resolved

erlend-aasland added 2 commits July 24, 2023 15:05

Pull in main
8791822

Improve accuracy of test_cli_converters
a4adb70

erlend-aasland requested a review from sobolevn July 24, 2023 15:47

sobolevn approved these changes Jul 24, 2023
View reviewed changes

erlend-aasland merged commit 83a2837 into python:mainJul 24, 2023

bedevere-bot removed the awaiting core review label Jul 24, 2023

erlend-aasland deleted the clinic/coverage-cli branch July 24, 2023 18:49

erlend-aasland added the needs backport to 3.11 only security fixes label Jul 24, 2023

erlend-aasland added the needs backport to 3.12 only security fixes label Jul 24, 2023

bedevere-bot removed the needs backport to 3.12 only security fixes label Jul 24, 2023

bedevere-bot removed the needs backport to 3.11 only security fixes label Jul 24, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

gh-106368: Increase Argument Clinic CLI test coverage#107156

gh-106368: Increase Argument Clinic CLI test coverage #107156

Uh oh!

erlend-aasland commented Jul 23, 2023•
edited
Loading

Uh oh!

erlend-aasland commented Jul 23, 2023

Uh oh!

erlend-aasland commented Jul 23, 2023

Uh oh!

sobolevn left a comment

Uh oh!

Uh oh!

Uh oh!

erlend-aasland commented Jul 24, 2023

Uh oh!

sobolevn left a comment

Uh oh!

Uh oh!

sobolevn left a comment

Uh oh!

miss-islington commented Jul 24, 2023

Uh oh!

miss-islington commented Jul 24, 2023

Uh oh!

bedevere-bot commented Jul 24, 2023

Uh oh!

bedevere-bot commented Jul 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

gh-106368: Increase Argument Clinic CLI test coverage#107156

gh-106368: Increase Argument Clinic CLI test coverage #107156

Uh oh!

Conversation

erlend-aasland commented Jul 23, 2023• edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

erlend-aasland commented Jul 23, 2023

Uh oh!

erlend-aasland commented Jul 23, 2023

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

erlend-aasland commented Jul 24, 2023

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sobolevn left a comment

Choose a reason for hiding this comment

Uh oh!

miss-islington commented Jul 24, 2023

Uh oh!

miss-islington commented Jul 24, 2023

Uh oh!

bedevere-bot commented Jul 24, 2023

Uh oh!

bedevere-bot commented Jul 24, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

erlend-aasland commented Jul 23, 2023•
edited
Loading