Switch to runsettings for helix tests #24503
Conversation
ghost
added
the
area-infrastructure
|
runtime version of runsettings that I based ours off of: https://raw.githubusercontent.com/dotnet/runtime/9cb582dc73dff1aaf8cd60b8e4a68ee6ea46f66b/eng/testing/.runsettings |
|
I see |
|
That looks like a template. I see variable placeholders in there. |
|
Right, I nuked all the variables since we pass in our filter dynamically today, we could generate the runsettings with the right arguments before we upload the job, but that seems like overkill... and would be a pain for the env vars: |
| @@ -247,7 +247,7 @@ public async Task<int> RunTestsAsync() | |||
| { | |||
| // Timeout test run 5 minutes before the Helix job would timeout | |||
| var cts = new CancellationTokenSource(Options.Timeout.Subtract(TimeSpan.FromMinutes(5))); | |||
| var commonTestArgs = $"test {Options.Target} --logger:xunit --logger:\"console;verbosity=normal\" --blame \"CollectHangDump;TestTimeout=5m\""; | |||
| var commonTestArgs = $"test {Options.Target} -s .runsettings"; | |||
There was a problem hiding this comment.
Can we go back to dotnet vstest❔ That accepts --Settings|/Settings:<Settings File>.
There was a problem hiding this comment.
I'm not positive but isn't test a larger wrapper for vstest w/ more moving parts❔ I was surprised we switched to test in #24489.
There was a problem hiding this comment.
I believe we only switched to vstest because back in the day test didn't take a dll, from other conversations I believe they both end up being wrappers for the same thing underneath the hood so it doesn't really matter...
|
@dougbu sounds like the only value of switching to runsettings, is if we were to use the same settings for local developer runs, which makes this more of a larger change to be valueable, we'd have to switch to using runsettings during local dotnet test when developers are running tests locally. I guess this can be something for DOI, but there's no benefit at all to using runsettings otherwise just on our helix runs. |
|
This is more or less required for runtime, since they generate runsettings to point at a custom runtime that they test against, we don't have any of that complexity so we aren't necessarily needing to do this as a result |
|
Up to you @HaoK but, in general, if we don't need it, don't do it 😃 |
|
That was my leaning :) closing for now |
Followup to #24489
cc @davidfowl