Reenable Components E2E #9076

HaoK · 2019-04-04T19:19:57Z

No description provided.

HaoK · 2019-04-16T23:21:33Z

Looks looks like we are blocked until they push node.js to the helix queues (which they are waiting for after preview 4 stabailizes to do)... these are failing with missing npm errors now

Stack Trace:
   at Microsoft.AspNetCore.E2ETesting.SeleniumStandaloneServer.InitializeInstance(ITestOutputHelper output)
   at Microsoft.AspNetCore.E2ETesting.SeleniumStandaloneServer.GetInstanceAsync(ITestOutputHelper output) in /_/src/Shared/E2ETesting/SeleniumStandaloneServer.cs:line 77
   at Microsoft.AspNetCore.E2ETesting.BrowserFixture.CreateBrowserAsync(String context, ITestOutputHelper output) in /_/src/Shared/E2ETesting/BrowserFixture.cs:line 95
   at Microsoft.AspNetCore.E2ETesting.BrowserTestBase.InitializeAsync() in /_/src/Shared/E2ETesting/BrowserTestBase.cs:line 42
[xUnit.net 00:02:03.16]     Microsoft.AspNetCore.Components.E2ETest.Tests.RoutingTest.CanFollowLinkToOtherPageWithQueryString [FAIL]
[xUnit.net 00:02:03.16]       System.InvalidOperationException : Failed to launch the server.
[xUnit.net 00:02:03.16]       ExitCode: 1
[xUnit.net 00:02:03.16]       Captured output lines:
[xUnit.net 00:02:03.16]       
[xUnit.net 00:02:03.16]       'npm' is not recognized as an internal or external command,
[xUnit.net 00:02:03.17]       operable program or batch file.
[xUnit.net 00:02:03.17]       .
[xUnit.net 00:02:03.17]       Stack Trace:
[xUnit.net 00:02:03.17]            at Microsoft.AspNetCore.E2ETesting.SeleniumStandaloneServer.InitializeInstance(ITestOutputHelper output)
[xUnit.net 00:02:03.17]         /_/src/Shared/E2ETesting/SeleniumStandaloneServer.cs(77,0): at Microsoft.AspNetCore.E2ETesting.SeleniumStandaloneServer.GetInstanceAsync(ITestOutputHelper output)
[xUnit.net 00:02:03.17]         /_/src/Shared/E2ETesting/BrowserFixture.cs(95,0): at Microsoft.AspNetCore.E2ETesting.BrowserFixture.CreateBrowserAsync(String context, ITestOutputHelper output)
[xUnit.net 00:02:03.17]         /_/src/Shared/E2ETesting/BrowserTestBase.cs(42,0): at Microsoft.AspNetCore.E2ETesting.BrowserTestBase.InitializeAsync()
[xUnit.net 00:02:03.18]     Microsoft.AspNetCore.Components.E2ETest.Tests.RoutingTest.ClickingAnchorWithNoHrefShouldNotNavigate [FAIL]
[xUnit.net 00:02:03.18]       System.InvalidOperationException : Failed to launch the server.
[xUnit.net 00:02:03.18]       ExitCode: 1
[xUnit.net 00:02:03.18]       Captured output lines:
[xUnit.net 00:02:03.18]       
[xUnit.net 00:02:03.18]       'npm' is not recognized as an internal or external command,
[xUnit.net 00:02:03.18]       operable program or batch file.

HaoK · 2019-04-22T19:52:50Z

Getting a little farther now: error is with selenium-standalone not being found now.

Is this something that should get installed as part of the run or something that we need to pull down before running these Component E2E tests?

@javiercn @SteveSandersonMS

https://mc.dot.net/#/user/aspnetcore/pr~2Faspnet~2Faspnetcore/ci/20190422.4/workItem/Microsoft.AspNetCore.Components.E2ETests-netcoreapp3.0/wilogs

[xUnit.net 00:01:01.96]       > [email protected] selenium-standalone C:\dotnetbuild\work\c83b73fd-01bd-4958-a429-55c880f0cd0a\Work\7463a409-fa45-476d-9616-7d1cc52f6610\Exec
[xUnit.net 00:01:01.96]       > selenium-standalone "start" "--" "-port" "50466"
[xUnit.net 00:01:01.96]       
[xUnit.net 00:01:01.96]       'selenium-standalone' is not recognized as an internal or external command,
[xUnit.net 00:01:01.96]       operable program or batch file.
[xUnit.net 00:01:01.96]       npm ERR! code ELIFECYCLE
[xUnit.net 00:01:01.96]       npm ERR! errno 1
[xUnit.net 00:01:01.96]       npm ERR! [email protected] selenium-standalone: `selenium-standalone "start" "--" "-port" "50466"`
[xUnit.net 00:01:01.96]       npm ERR! Exit status 1
[xUnit.net 00:01:01.96]       npm ERR! 
[xUnit.net 00:01:01.96]       npm ERR! Failed at the [email protected] selenium-standalone script.
[xUnit.net 00:01:01.96]       npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
[xUnit.net 00:01:01.96]       npm WARN Local package.json exists, but node_modules missing, did you mean to install?
[xUnit.net 00:01:01.96]       
[xUnit.net 00:01:01.96]       npm ERR! A complete log of this run can be found in:
[xUnit.net 00:01:01.96]       npm ERR!     C:\Users\runner\AppData\Roaming\npm-cache\_logs\2019-04-22T10_26_39_530Z-debug.log
[xUnit.net 00:01:01.96]

javiercn · 2019-04-22T21:57:49Z

It’s installed at build time with yarn install

HaoK · 2019-04-22T22:00:08Z

So for the helix machines which aren't building the app, we'll need to add some custom commands to install things?

HaoK · 2019-04-22T22:01:46Z

Is there a simple runtime command we can do that's equivalent to this?

https://github.com/aspnet/AspNetCore/blob/5889aea06cc7ed5c2f743e2fd8ed8abcb7c2d957/src/Middleware/NodeServices/src/Microsoft.AspNetCore.NodeServices.csproj#L24

Like before running the tests, do a 'yarn install packages.json' in the directory?

javiercn · 2019-04-22T22:06:01Z

You need the packages.json along with the yarn.lock and you should have yarn installed globally.

HaoK · 2019-04-22T22:09:00Z

So include packages.json and yarn.lock, and then run "yarn install" before running the tests should do the trick?

src/Shared/E2ETesting/SeleniumStandaloneServer.cs

HaoK · 2019-04-27T04:10:16Z

@javiercn before I dig too much farther, these tests should run on OSX and linux right?

Currently seeing

 System.ComponentModel.Win32Exception : No such file or directory
Stack Trace:
   at System.Diagnostics.Process.ForkAndExecProcess(String filename, String[] argv, String[] envp, String cwd, Boolean redirectStdin, Boolean redirectStdout, Boolean redirectStderr, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec)
   at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo)
   at System.Diagnostics.Process.Start()
   at System.Diagnostics.Process.Start(ProcessStartInfo startInfo)
   at Microsoft.AspNetCore.E2ETesting.SeleniumStandaloneServer.InitializeInstance(ITestOutputHelper output) in /_/src/Shared/E2ETesting/SeleniumStandaloneServer.cs:line 130
   at Microsoft.AspNetCore.E2ETesting.SeleniumStandaloneServer.GetInstanceAsync(ITestOutputHelper output) in /_/src/Shared/E2ETesting/SeleniumStandaloneServer.cs:line 77
   at Microsoft.AspNetCore.E2ETesting.BrowserFixture.CreateBrowserAsync(String context, ITestOutputHelper output) in /_/src/Shared/E2ETesting/BrowserFixture.cs:line 95
   at Microsoft.AspNetCore.E2ETesting.BrowserTestBase.InitializeAsync() in /_/src/Shared/E2ETesting/BrowserTestBase.cs:line 42
Failed   Microsoft.AspNetCore.Components.E2ETest.ServerExecutionTests.ServerEventTest.PreventDefault_AppliesToFormOnSubmitHandlers
Error Message:

It seems to be weird getting Win32 exceptions on Non Windows machines...?

javiercn · 2019-04-27T13:43:45Z

We only run them on windows, but we have plans to run them on Linux and MAC too. I believe @ryanbrandenburg was working on that.

ryanbrandenburg · 2019-04-29T16:27:19Z

Actually, I found that all of our tests actually already run on Linux and Mac. That's weird to me considering it's specifically disabled. My preliminary guess is that a property got messed up when we added the RunTemplatesTests property to split the tests into different tasks.

javiercn · 2019-04-29T16:47:12Z

Actually, I found that all of our tests actually already run on Linux and Mac. That's weird to me considering it's specifically disabled. My preliminary guess is that a property got messed up when we added the RunTemplatesTests property to split the tests into different tasks.

No they don't. The Selenium part only runs on Windows. Other parts run cross-plat. (Create/Restore/Build/Publish, etc).

HaoK · 2019-04-29T16:51:23Z

@MattGal looks like something got borked after we install yarn on windows, any ideas what happened to this job?

https://mc.dot.net/#/user/aspnetcore/pr~2Faspnet~2Faspnetcore/ci/20190427.5/workItem/Microsoft.AspNetCore.Components.E2ETests-netcoreapp3.0/wilogs

Windows.10.Amd64.ClientRS4.VS2017.Open
Details from Job a096b9ca-762c-429a-902c-764f5c662eba
ExitCode: 0 
Ran on Machine: a000I34 
Console 
 Executed on a000I34

C:\dotnetbuild\work\a096b9ca-762c-429a-902c-764f5c662eba\Work\c24d1c39-8242-4683-a274-4f318e18e6b3\Exec>npm i yarn 
npm notice created a lockfile as package-lock.json. You should commit this file.
+ [email protected]
added 1 package and audited 1 package in 2.719s
found 0 vulnerabilities

The other non windows queues were able to continue after the yarn install.

MattGal · 2019-04-29T18:22:12Z

@HaoK this happened because your command caused exit to happen at that point with code 0. I would expect that something is wrong with how the script being called is invoked which is promoting some other thing's exit /b 0 or equivalent into an exit. I'd first suggest you improve logging around how this is being invoked, then use the local or VM-based repro tools if this doesn't make the problem more obvious.

MattGal · 2019-04-29T18:32:04Z

I tried this Kusto query:

WorkItems | where Started > now(-10d) | where FriendlyName == "Microsoft.AspNetCore.Components.E2ETests-netcoreapp3.0" | join kind= inner (
   Jobs 
| where QueueName startswith "Windows"
| order by Started desc | project  QueueName , JobId, Build, Type, Source 
) on JobId  
| order by Finished desc

and I find no evidence that this worked on "some other Windows queue" or was ever run there.

I suggest trying the guidance here: npm/npm#2938.

I was able to repro this and fix it by just adding a call, e.g. try these versions of it in a local .bat file

@echo off

REM Shows the problem you were talking about
echo BEFORE
npm i yarn 
echo AFTER

REM Gets past the problem you were talking about
echo BEFORE
call npm i yarn 
echo AFTER

HaoK · 2019-04-29T18:41:06Z

Thanks @MattGal yeah the other queues were non windows that weren't hitting this issue

HaoK · 2019-04-30T19:35:14Z

Almost there it looks like, @javiercn hopefully the last issue is just another case of resolving the content root properly on helix.

Its looking for the sources using repo root which isn't there on helix

[xUnit.net 00:00:19.36]       System.ArgumentException : The content root 'F:\workspace\_work\1\s\src\Components\test\testassets\BasicTestApp' does not exist.

Looks like the logic is coming from here https://github.com/aspnet/AspNetCore/blob/helix-comp/src/Components/test/E2ETest/Infrastructure/ServerFixtures/ServerFixture.cs#L31 which is using the props setting the content root here: https://github.com/aspnet/AspNetCore/blob/5889aea06cc7ed5c2f743e2fd8ed8abcb7c2d957/src/Shared/E2ETesting/E2ETesting.targets#L93

Are you ok if I add a if helix branch which just hardcodes the content root to AppContext.BaseDirectory + "/testassets/AppName" which we can just include everything testassets in the helix payload

Set SeleniumProcessTrackingFolder Include package.json Yarn install Update Microsoft.AspNetCore.Components.E2ETests.csproj Update Microsoft.AspNetCore.Components.E2ETests.csproj Install yarn first Update Microsoft.AspNetCore.Components.E2ETests.csproj Update SeleniumStandaloneServer.cs Update runtests.cmd Update SeleniumStandaloneServer.cs Update SeleniumStandaloneServer.cs Update SeleniumStandaloneServer.cs Update SeleniumStandaloneServer.cs Only target VS windows helix queue Only run on windows, fix npm call Update Microsoft.AspNetCore.Components.E2ETests.csproj Update Microsoft.AspNetCore.Components.E2ETests.csproj Try resolving content root path on helix to current dir Strip out extra assembly name gunk from content root publish test assets use AppContext.BaseDirectory

HaoK · 2019-06-07T18:27:08Z

Replaced by #10997

Eilon added the area-mvc Includes: MVC, Actions and Controllers, Localization, CORS, most templates label Apr 4, 2019

HaoK force-pushed the helix-comp branch from 3f887e4 to eb0afc8 Compare April 15, 2019 19:00

HaoK added the blocked The work on this issue is blocked due to some dependency label Apr 16, 2019

HaoK force-pushed the helix-comp branch from 482fc34 to 8c38fbb Compare April 22, 2019 01:51

javiercn reviewed Apr 26, 2019

View reviewed changes

src/Shared/E2ETesting/SeleniumStandaloneServer.cs Outdated Show resolved Hide resolved

HaoK removed the blocked The work on this issue is blocked due to some dependency label Apr 29, 2019

HaoK force-pushed the helix-comp branch from b832bb9 to 5212caf Compare June 7, 2019 18:16

isaacrlevin and others added 7 commits June 7, 2019 11:19

Added ILogger and cleanup

bddec23

Update _ValidationScriptsPartial.cshtml

28bb670

Update Index.cshtml.cs

5dc0b86

Update Privacy.cshtml.cs

84dbf93

Update OidcConfigurationController.cs

7744e92

Update OidcConfigurationController.cs

9d4e9ca

Update OidcConfigurationController.cs

0ed8189

isaacrlevin and others added 6 commits June 7, 2019 11:19

Update SampleDataController.cs

0cf8c0c

Update Error.cshtml.cs

a90fc1d

Update Error.cshtml.cs

604e1d4

Update SampleDataController.cs

fd5fc50

Fix inconsistent casing of LoginDisplay component (#10966)

e9ea3ad

HaoK force-pushed the helix-comp branch from 5926418 to cf8894d Compare June 7, 2019 18:19

HaoK closed this Jun 7, 2019

HaoK deleted the helix-comp branch December 5, 2019 16:56

github-actions bot locked and limited conversation to collaborators Dec 8, 2023

Reenable Components E2E #9076

Reenable Components E2E #9076

Uh oh!

Conversation

HaoK commented Apr 4, 2019

Uh oh!

HaoK commented Apr 16, 2019

Uh oh!

HaoK commented Apr 22, 2019

Uh oh!

javiercn commented Apr 22, 2019

Uh oh!

HaoK commented Apr 22, 2019

Uh oh!

HaoK commented Apr 22, 2019

Uh oh!

javiercn commented Apr 22, 2019

Uh oh!

HaoK commented Apr 22, 2019

Uh oh!

Uh oh!

HaoK commented Apr 27, 2019

Uh oh!

javiercn commented Apr 27, 2019

Uh oh!

ryanbrandenburg commented Apr 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

javiercn commented Apr 29, 2019

Uh oh!

HaoK commented Apr 29, 2019

Uh oh!

MattGal commented Apr 29, 2019

Uh oh!

MattGal commented Apr 29, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

HaoK commented Apr 29, 2019

Uh oh!

HaoK commented Apr 30, 2019

Uh oh!

HaoK commented Jun 7, 2019

Uh oh!

Uh oh!

ryanbrandenburg commented Apr 29, 2019 •

edited

Loading

MattGal commented Apr 29, 2019 •

edited

Loading