Start skipping helix ready tests on azdo runs #17109
Conversation
github-actions
bot
added
area-infrastructure
|
Test count with this change goes to: 25889 total reported on azdo compared to 115806 today |
HaoK
requested review from
analogrelay,
halter73,
jkotalik,
SteveSandersonMS,
Tratcher and
a team
as code owners
|
Are all of the tests that are now skipped on azdo part of a blocking helix run, or are some of them still only run in the optional helix run? They should be required on helix before being disabled on azdo, not just optional. |
| @@ -5,6 +5,7 @@ | |||
|
|
|||
| <!-- Tests do not work on Helix or when bin/ directory is not in project directory due to undeclared dependency on test content. --> | |||
| <BuildHelixPayload>false</BuildHelixPayload> | |||
| <SkipTests>false</SkipTests> | |||
There was a problem hiding this comment.
Can't you just query BuildHelixPayload?
There was a problem hiding this comment.
Possibly, what are you suggesting?
There was a problem hiding this comment.
Is it possible to define SkipTests globally based on BuildHelixPayload?
There was a problem hiding this comment.
I'm not a msbuild pro at all, my naive understanding is that its just evaluating files in order, so there's no way to default it dynamically globally so when its turned off later in a particular project it just works, which is why I just went with blasting it both flags to false. There's no way to defer evaluation by reference like that, I don't think.
There was a problem hiding this comment.
@HaoK suggest you move the setting to the root Directory.Build.targets file (after line 110 or so, where $(BuildHelixPayload) gets its default) and change the condition to
<SkipTests Condition="'$(SkipHelixReadyTests)' == 'true' AND '$(BuildHelixPayload)' == 'true'">true</SkipTests>
There was a problem hiding this comment.
awesome this is much better!
|
The ubuntu queue is a blocking PR check, I believe there are some windows only tests that are being run in the optional helix queue still, we probably should make all (or some) windows queues required as part of this switch |
|
Adding Win10 and OSX1014 to the mandatory helix green checks to match what we had in azdo |
|
@aspnet/build everything seems to be passing now, I think this PR is ready for final review |
|
BTW this looks fine to me but I'd like someone more familiar w/ Helix to review more closely |
|
FYI |
|
Now that #18274 has been merged, logging should be available on helix runs for test bugging, we should be ready for final sign on off on this PR @dotnet/aspnet-build |
amcasey
added
area-networking
Skips all tests on azdo by default, selectively enables test projects that currently don't work on helix (
BuildHelixPayload = false).Note: we only run ubuntu and windows helix queues on every PR, so we would no longer be running mac OS tests as frequently
Per discussion in build sync: initial plan will be:
Required for every PR: Windows 10 and Ubuntu
Rolling builds for all other helix queues