Partest for Dotty with pos tests and neg tests with error count #482
Conversation
| @@ -79,7 +83,9 @@ object DottyBuild extends Build { | |||
|
|
|||
| tuning ::: agentOptions ::: travis_build ::: fullpath | |||
| } | |||
| ) | |||
| ) ++ (if (isTravisBuild) Seq() else addCommandAlias("test", "partest")) | |||
There was a problem hiding this comment.
why do we need this alias? why not have both test and partest?
There was a problem hiding this comment.
The test key only generates partest files, it doesn't run the tests. So the alias prevents people from having to remember to use partest instead of test.
There was a problem hiding this comment.
Can we leave the chance to have both? Ie to be able to either execute tests with partest or with JUnit from sbt?
There was a problem hiding this comment.
the main reason is that I use javaagent to attach to JUnit tests from sbt (see agentOptions), and attaching to partest doesn't make much sense.
There was a problem hiding this comment.
I see. Does this need to be the test target or would it work with another target (say nonpartest)? I briefly looked at this already, but couldn't get it to work as I couldn't use test frameworks from my own key or set it up the same as Test and test... But I'll try again.
Alternatively, if you need to edit the Build.scala anyway to uncomment agentOptions, I could also add a flag to set agentOptions and use JUnit in that case. That would be much simpler, but wouldn't give a way to run JUnit without reload.
There was a problem hiding this comment.
Making test mean test feals clearer to me, as travis could also benefit from it(it would run test, not partest).
Silently replacing default task of sbt feels like a very unintuitive action for newcomers, who would naturally expect test to run tests in the same way as travis does it. This cryptic detail would make it harder for newcomers to reproduce failures seen on travis.
If's it's indeed impossible or very hard to achieve, we can live with this.
|
I figured out how to keep the two independent targets:
It's slightly hacky, but should be good enough (not sure what concurrency guarantees are made with respect to deleting a file, in the worst case running partest;test would run partest twice) The PR is now marked as having merge conflicts. Do I still rebase despite the warnings about rewriting history that has been pushed? And then create a new PR? |
|
I suggest rebasing and push-forcing this branch, that will update the PR and the comments on the PR diff won't be lost (comments on individual commits are lost when you push-force a PR, but there isn't any in this PR so you're safe). |
vsalvis
force-pushed
the
vsalvis-partest2
branch
from
c0f5cd5 to
fb92b97
Compare
|
Thanks, done! Also updated the PR description. |
vsalvis
force-pushed
the
vsalvis-partest2
branch
from
fb92b97 to
8d08915
Compare
|
@vsalvis Nice Idea! |
|
Done, thanks for the pointer. Travis failure is flaky (failed to download sbt-launcher), the last commit passes locally. |
Partest for Dotty with pos tests and neg tests with error count
Adding partest infrastructure to Dotty, currently for pos and neg tests. The sources are generated from tests.scala, which is still a regular JUnit test suite.
sbt testruns JUnit as before,sbt partestnow generates the sources and runs partest.I couldn't get everything to run through on travis, which fails at a random partest test with
java.lang.RuntimeException: Nonzero exit code returned from runner: 137so for now travis will keep doingtestinstead ofpartest.For eclipse/IntelliJ tests.scala can be used as before. But don't run JUnit tests from the IDE at the same time as running
sbt partest. Firstly, in the IDE all tests will pass (they're generating files instead of running the tests) and secondly, partest in the console might fail because you're messing with the directory structure.