Add pyright testcases / regression tests to the runtests script #10002
Conversation
There was a problem hiding this comment.
Makes sense! A few minor points below.
Also, if there are no test cases for a stubs package, on my machine (without pyright installed) a message is printed to the terminal saying that the regression tests for pyright were skipped. But that's not really accurate if there are no regression tests to skip!
Skipped tests means no work was done towards determining whether it would fail or pass on CI. If the npx command fails (ie not installed), the test is skipped. So it's intended. That being said, the lack of test_cases is an automatic success (0/0 tests passed). And we can check for that early. Imo it would also be faster to do that additional check rather than letting pyright setup and run on 0 files. I also just thought that I didn't try or account for stdlib. |
Co-authored-by: Alex Waygood <[email protected]>
|
Updated to first check for the existance of the |
There was a problem hiding this comment.
Looks good!
This still doesn't do what I'd expect:
(.venv) typeshed>python scripts/runtests.py stdlib
usage: runtests.py [-h] [--run-stubtest] path
runtests.py: error: 'path' argument should be in format <folder>/<stub>.
But we have that behaviour on main as well; fixing it can wait for another PR
No description provided.