Skip to content

regression-tests: run tests using bash #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

hugolm84
Copy link

Convenience script for running regression tests on Linux. Current results on Ubuntu and gcc11:

./run-tests.sh
TRANSPILE:
34 SUCCESS
8 FAILED
./test-results/mixed-lifetime-safety-pointer-init-3.cpp2
./test-results/mixed-initialization-safety-1.cpp2
./test-results/mixed-lifetime-safety-pointer-init-2.cpp2
./test-results/mixed-initialization-safety-2.cpp2
./test-results/mixed-lifetime-safety-pointer-init-1.cpp2
./test-results/pure2-bounds-safety-pointer-arithmetic-error.cpp2
./test-results/pure2-lifetime-safety-pointer-init-1.cpp2
./test-results/pure2-lifetime-safety-reject-null.cpp2
COMPILE:
33 SUCCESS
1 FAILED
./test-results/mixed-captures-in-expressions-and-postconditions.cpp
./run-tests.sh: line 26: 184219 Segmentation fault      (core dumped) ./$filename >> ./$filename-output 2>&1
./run-tests.sh: line 26: 184231 Aborted                 (core dumped) ./$filename >> ./$filename-output 2>&1
RUN:
31 SUCCESS
2 FAILED
./test-results/mixed-lifetime-safety-and-null-contracts.cpp.out
./test-results/mixed-bounds-safety-with-assert.cpp.out

@hsutter
Copy link
Owner

hsutter commented Oct 1, 2022

Thanks. I'm slowly overhauling the regressing testing to run the tests, and I plan to also do that in bach. I'll defer this until then, I won't get to it immediately but perhaps after the Kona meeting. Again, thanks!

@hsutter hsutter self-assigned this Oct 1, 2022
@hsutter hsutter added the defer label Oct 1, 2022
@mordante mordante mentioned this pull request Oct 9, 2022
@PastaPastaPasta
Copy link

I was actually just beginning to work on a very similar PR, glad I saw this one before I got too into it. I would definitely propose merging this now, and potentially reverting it later when the regression tests are overhauled.

@hsutter
Copy link
Owner

hsutter commented Dec 13, 2022

I've now finished adding support for actually running the regression tests using MSVC, GCC, and Clang. Please see the new stuff under /regression-tests including checked-in results of executing the tests.

I know there's more that could be done, including CI, but for now I'm happy with just running four batch/shell scripts by hand (one to put all the Cpp2 tests through cppfront, and then one each per compiler to build and run the Cpp1 output).

Thanks!

@hsutter hsutter closed this Dec 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants