[release/3.1] Stop building on Ubuntu 18.04 #43666

dougbu · 2022-08-31T21:17:16Z

use 20.04 instead

nit: Remove Ubuntu version from one CI job display name

dougbu · 2022-08-31T21:23:01Z

@halter73 @Tratcher I bet PR validation here is going to fail similarly to how my manual #20220830.17 build failed: A number of tests (TlsHandshakeRejectsTlsLessThan12, DevCertWithInvalidPrivateKeyProducesCustomWarning, ClientAttemptingToUseUnsupportedProtocolIsLoggedAsDebug, and DoesNotSupportTls10) failed in both attempts there.

For example,

Error message
Assert.Throws() Failure\nExpected: typeof(System.IO.IOException)
Actual:   typeof(System.Security.Authentication.AuthenticationException): Authentication failed, see inner exception.
---- System.Security.Authentication.AuthenticationException : Authentication failed, see inner exception.
-------- Interop+OpenSsl+SslException : SSL Handshake failed with OpenSSL error - SSL_ERROR_SSL.
------------ Interop+Crypto+OpenSslCryptographicException : error:141E70BF:SSL routines:tls_construct_client_hello:no protocols available

Stack trace
at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ProcessAuthentication(LazyAsyncResult lazyResult, CancellationToken cancellationToken)
at System.Net.Security.SslStream.BeginAuthenticateAsClient(SslClientAuthenticationOptions sslClientAuthenticationOptions, CancellationToken cancellationToken, AsyncCallback asyncCallback, Object asyncState)
at System.Net.Security.SslStream.BeginAuthenticateAsClient(String targetHost, X509CertificateCollection clientCertificates, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation, AsyncCallback asyncCallback, Object asyncState)
at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__64_1(String arg1, X509CertificateCollection arg2, SslProtocols arg3, AsyncCallback callback, Object state)
at System.Threading.Tasks.TaskFactory`1.FromAsyncImpl[TArg1,TArg2,TArg3](Func`6 beginMethod, Func`2 endFunction, Action`1 endAction, TArg1 arg1, TArg2 arg2, TArg3 arg3, Object state, TaskCreationOptions creationOptions)
at System.Threading.Tasks.TaskFactory.FromAsync[TArg1,TArg2,TArg3](Func`6 beginMethod, Action`1 endMethod, TArg1 arg1, TArg2 arg2, TArg3 arg3, Object state, TaskCreationOptions creationOptions)
at System.Threading.Tasks.TaskFactory.FromAsync[TArg1,TArg2,TArg3](Func`6 beginMethod, Action`1 endMethod, TArg1 arg1, TArg2 arg2, TArg3 arg3, Object state)
at System.Net.Security.SslStream.AuthenticateAsClientAsync(String targetHost, X509CertificateCollection clientCertificates, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation)
[at Microsoft.AspNetCore.Server.Kestrel.InMemory.FunctionalTests.HttpsConnectionMiddlewareTests.<>c__DisplayClass15_0.<<DoesNotSupportTls10>b__3>d.MoveNext() in /_/src/Servers/Kestrel/test/InMemory.FunctionalTests/HttpsConnectionMiddlewareTests.cs:line 388](https://dev.azure.com/dnceng/internal/_git/23330b81-8bf6-4092-af7d-db8dfba063b8?path=%2F_%2Fsrc%2FServers%2FKestrel%2Ftest%2FInMemory.FunctionalTests%2FHttpsConnectionMiddlewareTests.cs&version=GBdougbu%2Fubuntu.20.04%2F3.1&_a=contents&line=388&lineEnd=389&lineStartColumn=1&lineEndColumn=1&lineStyle=plain)
--- End of stack trace from previous location where exception was thrown ---
----- Inner Stack Trace -----
at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ForceAuthentication(Boolean receiveFirst, Byte[] buffer, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslStream.ProcessAuthentication(LazyAsyncResult lazyResult, CancellationToken cancellationToken)
at System.Net.Security.SslStream.BeginAuthenticateAsClient(SslClientAuthenticationOptions sslClientAuthenticationOptions, CancellationToken cancellationToken, AsyncCallback asyncCallback, Object asyncState)
at System.Net.Security.SslStream.BeginAuthenticateAsClient(String targetHost, X509CertificateCollection clientCertificates, SslProtocols enabledSslProtocols, Boolean checkCertificateRevocation, AsyncCallback asyncCallback, Object asyncState)
at System.Net.Security.SslStream.<>c.<AuthenticateAsClientAsync>b__64_1(String arg1, X509CertificateCollection arg2, SslProtocols arg3, AsyncCallback callback, Object state)
at System.Threading.Tasks.TaskFactory`1.FromAsyncImpl[TArg1,TArg2,TArg3](Func`6 beginMethod, Func`2 endFunction, Action`1 endAction, TArg1 arg1, TArg2 arg2, TArg3 arg3, Object state, TaskCreationOptions creationOptions)
at System.Threading.Tasks.TaskFactory.FromAsync[TArg1,TArg2,TArg3](Func`6 beg

In at least one later branch, ClientAttemptingToUseUnsupportedProtocolIsLoggedAsDebug() looks for any Exception. Should I make the same change here❔

dougbu · 2022-08-31T21:29:36Z

/cc @BrennanConroy because the IOException ➡️ Exception change came in from #18584.

dougbu · 2022-08-31T21:34:23Z

I'm thinking at least part of what's went on in that PR related to making the thrown Exceptions consistent across platforms, at least where OpenSSL is used. It may be the case that the runtime team didn't backport their changes to 3.1.x. Is that right @wfurt and @karelz❔

dougbu · 2022-08-31T21:59:29Z

More information also available in https://dev.azure.com/dnceng/public/_build/results?buildId=1979965&view=ms.vss-test-web.build-test-results-tab&runId=50627370&paneView=debug

halter73 · 2022-08-31T22:06:22Z

@dougbu is the "[release/2.1]" in the title a typo? This is targeting release/3.1.

I don't think changing the exception type here matters much even in servicing. I doubt any real app to relies on AuthenticateAsClientAsync throwing an IOException instead of an AuthenticationException in these kinds of scenarios. The tests are just overly specific.

wfurt · 2022-08-31T22:07:26Z

I don't really remember @dougbu. 2.1 is out of support and 3.1 support ends in December. Running on Ubuntu 20.04 seems fine. 22.04 is whole different beast as it comes with OpenSSL 3.

dougbu · 2022-08-31T23:24:39Z

@dougbu is the "[release/2.1]" in the title a typo? This is targeting release/3.1.

Yeah, my bad. Updating the title…

I don't think changing the exception type here matters much even in servicing

I'll make the change in my next iteration

I don't really remember @dougbu. 2.1 is out of support and 3.1 support ends in December.

Thanks for thinking back. (Note everything used in ASP.NET 2.1.x applications remains fully supported on .NET Framework:tm:)

Running on Ubuntu 20.04 seems fine. 22.04 is whole different beast as it comes with OpenSSL 3.

In addition, 20.04 should remain in support until April 2030 😀

dougbu · 2022-08-31T23:53:04Z

@mthalman @crummel @lbussell which dotnet-buildtools/prereqs image should I use to replace mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-arm64-alpine-85814b7-20200327151156❔

I can't find anything w/ alpine in the name on https://mcr.microsoft.com/v2/dotnet-buildtools/prereqs/tags/list between ubuntu-18.04-cross-arm64-alpine-85814b7-20200327151156 (how nice, we were at the latest 18.04-based image 😀) and ubuntu-22.04-cross-arm-alpine-20220830132559-b016fdd. We'd like an Ubuntu 20.04-based image for our Linux Musl ARM64 build job but I'm not sure how to search for what we need.

dougbu · 2022-09-01T00:05:05Z

If it matters, we use the mcr.microsoft.com/dotnet/runtime-deps:2.1-alpine for our Linux Musl x64 build. Should we therefore consider mcr.microsoft.com/dotnet/runtime-deps:3.1.28-alpine3.15-arm64v8 or the alpine3.16 image❔ (I'm having trouble parsing the old Docker files to find which Alpine version we were getting.)

dougbu · 2022-09-01T02:19:10Z

/azp run

azure-pipelines · 2022-09-01T02:19:22Z

Azure Pipelines failed to run 1 pipeline(s).

mthalman · 2022-09-01T13:26:53Z

which dotnet-buildtools/prereqs image should I use to replace mcr.microsoft.com/dotnet-buildtools/prereqs:ubuntu-18.04-cross-arm64-alpine-85814b7-20200327151156❔

I can't find anything w/ alpine in the name on https://mcr.microsoft.com/v2/dotnet-buildtools/prereqs/tags/list between ubuntu-18.04-cross-arm64-alpine-85814b7-20200327151156 (how nice, we were at the latest 18.04-based image 😀) and ubuntu-22.04-cross-arm-alpine-20220830132559-b016fdd. We'd like an Ubuntu 20.04-based image for our Linux Musl ARM64 build job but I'm not sure how to search for what we need.

If there isn't an existing one, the expectation is that teams will submit PRs to https://github.com/dotnet/dotnet-buildtools-prereqs-docker to produce images that meet their needs. Since this is just a move from 18.04 to 20.04, this might be as simple as copy-and-paste and update the version number.

ghost · 2022-09-01T20:02:41Z

Hey @dotnet/aspnet-build, looks like this PR is something you want to take a look at.

dougbu · 2022-09-14T05:23:29Z

Internal test build #20220912.21 was successful, except for the DevCertWithInvalidPrivateKeyProducesCustomWarning failure addressed in the last commit here.

dougbu · 2022-09-14T05:25:54Z

src/Servers/Kestrel/test/InMemory.FunctionalTests/HttpsTests.cs

                        sslStream.AuthenticateAsClientAsync("127.0.0.1", clientCertificates: null,
                            enabledSslProtocols: SslProtocols.Tls,
                            checkCertificateRevocation: false));
                }
            }

            await loggerProvider.FilterLogger.LogTcs.Task.DefaultTimeout();
-            Assert.Equal(3, loggerProvider.FilterLogger.LastEventId);
+            Assert.Equal(1, loggerProvider.FilterLogger.LastEventId);


Not exactly sure why this needed to change. But it lines up w/ tests in later branches

wtgodbe

Infra changes seem fine

src/Servers/Kestrel/test/InMemory.FunctionalTests/HttpsTests.cs

- use 20.04 instead nit: Remove Ubuntu version from one CI job display name

…s<>()` - skip `TlsHandshakeRejectsTlsLessThan12` because TLS1.0 is not available on Ubuntu 20.04 - for the other 3 tests, `IOException` was too specific for the platform switch

- test does not behave consistently across platforms - test already removed in later branches - testing in this area has moved to CTI (manual testing)

dougbu changed the title ~~[release/2.1] Stop building on Ubuntu 18.04~~ [release/3.1] Stop building on Ubuntu 18.04 Aug 31, 2022

dougbu force-pushed the dougbu/ubuntu.20.04/3.1 branch from a71a544 to ffe5eb9 Compare September 1, 2022 00:45

dougbu closed this Sep 1, 2022

dougbu reopened this Sep 1, 2022

dougbu force-pushed the dougbu/ubuntu.20.04/3.1 branch from ffe5eb9 to 0deb156 Compare September 1, 2022 05:30

Pilchie added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Sep 1, 2022

dougbu force-pushed the dougbu/ubuntu.20.04/3.1 branch from 0abaf4f to 83ffcc9 Compare September 14, 2022 05:20

dougbu marked this pull request as ready for review September 14, 2022 05:21

dougbu requested review from Tratcher, halter73, BrennanConroy and a team as code owners September 14, 2022 05:21

dougbu enabled auto-merge (squash) September 14, 2022 05:24

dougbu commented Sep 14, 2022

View reviewed changes

wtgodbe approved these changes Sep 14, 2022

View reviewed changes

dougbu commented Sep 14, 2022

View reviewed changes

src/Servers/Kestrel/test/InMemory.FunctionalTests/HttpsTests.cs Outdated Show resolved Hide resolved

dougbu added 3 commits September 16, 2022 16:24

[release/3.1] Stop building on Ubuntu 18.04

a14ad6e

- use 20.04 instead nit: Remove Ubuntu version from one CI job display name

Skip TlsHandshakeRejectsTlsLessThan12 more and expand `Assert.Throw…

808b54a

…s<>()` - skip `TlsHandshakeRejectsTlsLessThan12` because TLS1.0 is not available on Ubuntu 20.04 - for the other 3 tests, `IOException` was too specific for the platform switch

Remove DevCertWithInvalidPrivateKeyProducesCustomWarning test

c9bde8b

- test does not behave consistently across platforms - test already removed in later branches - testing in this area has moved to CTI (manual testing)

dougbu force-pushed the dougbu/ubuntu.20.04/3.1 branch from 83ffcc9 to c9bde8b Compare September 16, 2022 23:29

dougbu merged commit dcf23c9 into dotnet:release/3.1 Sep 17, 2022

dougbu deleted the dougbu/ubuntu.20.04/3.1 branch September 17, 2022 00:27

dougbu mentioned this pull request Oct 4, 2022

[release/2.1] Switch to consolidated Windows build images #44347

Merged

[release/3.1] Stop building on Ubuntu 18.04 #43666

[release/3.1] Stop building on Ubuntu 18.04 #43666

Uh oh!

Conversation

dougbu commented Aug 31, 2022

Uh oh!

dougbu commented Aug 31, 2022

Uh oh!

dougbu commented Aug 31, 2022

Uh oh!

dougbu commented Aug 31, 2022

Uh oh!

dougbu commented Aug 31, 2022

Uh oh!

halter73 commented Aug 31, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wfurt commented Aug 31, 2022

Uh oh!

dougbu commented Aug 31, 2022

Uh oh!

dougbu commented Aug 31, 2022

Uh oh!

dougbu commented Sep 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dougbu commented Sep 1, 2022

Uh oh!

azure-pipelines bot commented Sep 1, 2022

Uh oh!

mthalman commented Sep 1, 2022

Uh oh!

ghost commented Sep 1, 2022

Uh oh!

dougbu commented Sep 14, 2022

Uh oh!

dougbu Sep 14, 2022

Choose a reason for hiding this comment

Uh oh!

wtgodbe left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

halter73 commented Aug 31, 2022 •

edited

Loading

dougbu commented Sep 1, 2022 •

edited

Loading