Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Make Python test suite fail if no tests ran #98903

Closed
vstinner opened this issue Oct 31, 2022 · 2 comments
Closed

Make Python test suite fail if no tests ran #98903

vstinner opened this issue Oct 31, 2022 · 2 comments
Labels
type-bug An unexpected behavior, bug, or error

Comments

@vstinner
Copy link
Member

vstinner commented Oct 31, 2022 •
edited by bedevere-bot

When the Python test suite is run with a typo in a test filename or a test method, it displays "NO TEST RUN" but the exit code is a success (exit code 0) and so the mistake can be easily missed. I propose to make the test suite fail with a non-zero exit code to highlight that something unusual happened.
@vstinner vstinner added the type-bug An unexpected behavior, bug, or error label Oct 31, 2022
vstinner added a commit to vstinner/cpython that referenced this issue Oct 31, 2022
@vstinner
pythongh-98903: Test suite fails with exit code 4 if no tests ran
b2f3ed8 
The Python test suite now fails wit exit code 4 if no tests ran. It
should help detecting typos in test names and test methods.

* Add "EXITCODE_" constants to Lib/test/libregrtest/main.py.
* Fix a typo: "NO TEST RUN" becomes "NO TESTS RAN"
@bedevere-bot bedevere-bot mentioned this issue Oct 31, 2022
Merged
@vstinner
Copy link
Member Author

vstinner commented Oct 31, 2022

Non-zero exit code should also help automation of running the Python test suite, with git bisect for example.

@vstinner
Copy link
Member Author

vstinner commented Nov 2, 2022

Fixed by c76db37

@vstinner vstinner closed this as completed Nov 2, 2022
vstinner added a commit that referenced this issue Nov 2, 2022
@vstinner
gh-98903: Test suite fails with exit code 4 if no tests ran (#98904)
c76db37 
The Python test suite now fails wit exit code 4 if no tests ran. It
should help detecting typos in test names and test methods.

* Add "EXITCODE_" constants to Lib/test/libregrtest/main.py.
* Fix a typo: "NO TEST RUN" becomes "NO TESTS RAN"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-bug An unexpected behavior, bug, or error
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@vstinner