Attempt to auto-publish vs.redist packages to orchestrated feed #20211

wtgodbe · 2020-03-26T21:25:43Z

Attempts to resolve https://github.com/dotnet/aspnetcore-internal/issues/3387

@mmitche is there a way I can run a test prodcon build of AspNetCore 2.1 to see if this works?

TODO: update https://devdiv.visualstudio.com/DevDiv/_git/DotNet-Orchestration-Utilities?path=%2Fsrc%2FFinal.VSFeedPublish.targets&version=GBmaster

wtgodbe · 2020-03-26T21:29:26Z

build/Publish.targets

@@ -276,6 +280,10 @@
        ManifestArtifactData="NonShipping=true"
        Condition="'%(PackageToPublish.Category)' != 'ship'" />

+      <PackageToPublishToTransport


PublishToTransportFeed is the target that runs when we're trying to publish to the orchestrated feed, so as long as I also update https://devdiv.visualstudio.com/DevDiv/_git/DotNet-Orchestration-Utilities?path=%2Fsrc%2FFinal.VSFeedPublish.targets&version=GBmaster, the orchestration should take care of copying the redist packages from the orchestrated feed to the VS feed

Another option would be to include these packages in @(FilesToPublish) i.e. changing the input GetFilesToPublish target. That has the advantage of placing the files in the \aspnetci drop too.

wtgodbe · 2020-03-26T21:31:37Z

This doesn't yet solve the issue of giving the vs.redist packages nonstable versions - working on that

dagood

Looks reasonable to me!

wtgodbe · 2020-03-26T21:41:50Z

I think this should also use non-stable versioning for the vs.redist package now without destabilizing anything else, but I'm not 100% sure because I haven't been able to successfully build locally yet 😆

dougbu

Looks good modulo one typo. Fire up a build on TeamCity if you wanna test it out -- publish attempts shouldn't hurt anything.

Directory.Build.props

dougbu · 2020-03-26T22:21:59Z

run a test prodcon build of AspNetCore 2.1

Unless you've updated the TC configuration to skip publishing for non-ProdCon builds, any build should do. And, if you did change the configuration, can temporarily change it back to be unconditional 😃

wtgodbe · 2020-03-26T22:24:37Z

Unless you've updated the TC configuration to skip publishing for non-ProdCon builds, any build should do. And, if you did change the configuration, can temporarily change it back to be unconditional 😃

Don't I need to provide some variables that would normally come from orchestration (e.g. the key for https://dotnetfeed.blob.core.windows.net/orchestrated-release-2-1/ ?

dougbu · 2020-03-26T22:27:25Z

Ah, you want to test to a later endpoint than I thought. I was just looking to see what the build tried to push.

wtgodbe · 2020-03-26T22:29:48Z

I was just looking to see what the build tried to push.

Well, that might be enough - I'll try queuing a team city build just in case

wtgodbe · 2020-03-26T22:35:49Z

I don't see a way to run a build against a custom branch in TeamCity 😦

wtgodbe · 2020-03-26T22:36:15Z

I could merge this then revert it, and run a build against the initial merge

wtgodbe · 2020-03-26T22:59:30Z

@JunTaoLuo helped me figure out how to change the branch, so I've got the build running now

build/Publish.targets

dougbu · 2020-03-27T02:20:28Z

build/Publish.targets

@@ -276,6 +280,10 @@
        ManifestArtifactData="NonShipping=true"
        Condition="'%(PackageToPublish.Category)' != 'ship'" />

+      <PackageToPublishToTransport


Another option would be to include these packages in @(FilesToPublish) i.e. changing the input GetFilesToPublish target. That has the advantage of placing the files in the \aspnetci drop too.

- should trigger a build that's more likely to succeed

wtgodbe · 2020-03-27T16:23:48Z

The build succeeded, but didn't attempt to publish the redist packages because it didn't pass PublishType=blob (and therefore didn't run the PublishToTransportFeed target). I'll queue another one w/ that param

wtgodbe · 2020-03-27T17:37:57Z

Here's the Dotnet-Orchestration-Utilities PR, @dagood PTAL: https://devdiv.visualstudio.com/DevDiv/_git/DotNet-Orchestration-Utilities/pullrequest/238164?_a=overview

…/aspnetcore into wtgodbe/AutoPublishRedist

wtgodbe · 2020-03-27T19:08:42Z

Build failed before it could be informative because we didn't pass the account name or key for the orchestrated feed. Instead I'm gonna just print what we would try to publish in the next build (I'll undo 3a5b333 before merging)

wtgodbe · 2020-03-27T19:10:16Z

@dougbu I have some concern about using FilesToPublish because I believe not everything in that ItemGroup is winding up in the orchestrated feed. I'll confirm with the build I have running now

wtgodbe · 2020-03-27T19:44:46Z

Looks like the outage is impacting my val build 😢 hopefully it's fixed soon, or at least by monday

JunTaoLuo

LGTM

wtgodbe · 2020-03-31T16:18:14Z

Not working yet, the redist packages aren't being included in PackageToPublishToTransport. Investigating why.

wtgodbe · 2020-03-31T20:29:32Z

I merged https://devdiv.visualstudio.com/DevDiv/_git/DotNet-Orchestration-Utilities/pullrequest/238164?_a=overview

wtgodbe · 2020-03-31T21:14:15Z

build/Publish.targets

@@ -174,6 +175,10 @@
      <PackageToPublish Include="%(_SymbolsPackageArtifactInfo.ArtifactPath)" Category="symbols" IsSymbolsPackage="true" />
    </ItemGroup>

+    <ItemGroup>
+      <RedistPackageToPublish Include="$(DependencyAssetsDir)(SharedFrameworkRedistPackageId)*" />


Well don't I have an egg on my face... This isn't working because there's no '$' before (SharedFrameworkRedistPackageId) 🤦‍♂

wtgodbe · 2020-04-01T16:31:36Z

Ok, this should be ready for review - the most recent TeamCity build confirmed it would attempt to publish the redist packages (w/ non-stable versions):

[17:28:04][Step 4/6] Publishing the following packages: D:\b\w\98f319efba687834/.deps\packages\Microsoft.AspNetCore.All.2.1.18.nupkg;D:\b\w\98f319efba687834/.deps\packages\Microsoft.AspNetCore.App.2.1.18.nupkg;D:\b\w\98f319efba687834/.deps\packages\Microsoft.DotNet.Web.Client.ItemTemplates.2.1.18.nupkg;D:\b\w\98f319efba687834/.deps\packages\Microsoft.DotNet.Web.ItemTemplates.2.1.18.nupkg;D:\b\w\98f319efba687834/.deps\packages\Microsoft.DotNet.Web.ProjectTemplates.2.1.2.1.18.nupkg;D:\b\w\98f319efba687834/.deps\packages\Microsoft.DotNet.Web.Spa.ProjectTemplates.2.1.2.1.18.nupkg;D:\b\w\98f319efba687834/.deps\packages\Microsoft.AspNetCore.Hosting.WebHostBuilderFactory.Sources.2.1.18.nupkg;D:\b\w\98f319efba687834/.deps\packages\Microsoft.AspNetCore.AspNetCoreModuleV1.2.1.18.nupkg;D:\b\w\98f319efba687834/.deps\packages\Microsoft.AspNetCore.AspNetCoreModuleV1.2.1.18.symbols.nupkg;D:\b\w\98f319efba687834/.deps\packages\runtime.win-x64.Microsoft.AspNetCore.All.2.1.18.symbols.nupkg;D:\b\w\98f319efba687834/.deps\packages\runtime.win-x64.Microsoft.AspNetCore.App.2.1.18.symbols.nupkg;D:\b\w\98f319efba687834/.deps\packages\runtime.win-x86.Microsoft.AspNetCore.All.2.1.18.symbols.nupkg;D:\b\w\98f319efba687834/.deps\packages\runtime.win-x86.Microsoft.AspNetCore.App.2.1.18.symbols.nupkg;D:\b\w\98f319efba687834/.deps\packages\runtime.linux-musl-x64.Microsoft.AspNetCore.All.2.1.18.symbols.nupkg;D:\b\w\98f319efba687834/.deps\packages\runtime.linux-musl-x64.Microsoft.AspNetCore.App.2.1.18.symbols.nupkg;D:\b\w\98f319efba687834/.deps\packages\runtime.linux-x64.Microsoft.AspNetCore.All.2.1.18.symbols.nupkg;D:\b\w\98f319efba687834/.deps\packages\runtime.linux-x64.Microsoft.AspNetCore.App.2.1.18.symbols.nupkg;D:\b\w\98f319efba687834.deps\assets\VS.Redist.Common.AspNetCore.SharedFramework.x64.2.1.2.1.18-servicing-31339.nupkg;D:\b\w\98f319efba687834.deps\assets\VS.Redist.Common.AspNetCore.SharedFramework.x86.2.1.2.1.18-servicing-31339.nupkg

wtgodbe · 2020-04-01T16:31:58Z

@dougbu @JunTaoLuo any last concerns?

wtgodbe · 2020-04-01T16:36:42Z

I believe I've reset everything in teamcity to the way it was (I only changed VCSBranches & IsFinalBuild), but it'd be great if you guys could take a glance at the project settings to see if it looks like I missed anything

dougbu · 2020-04-01T17:04:15Z

it'd be great if you guys could take a glance at the project settings to see if it looks like I missed anything

You can double-check things too. Compare the public and private build configurations and confirm the only differences are obvious eg. related to signing.

dougbu

Smallish bits to change before merging…

Directory.Build.props

build/Publish.targets

Directory.Build.props

build/Publish.targets

…/aspnetcore into wtgodbe/AutoPublishRedist

wtgodbe · 2020-04-01T17:36:42Z

@dougbu I pushed a commit responding to your feedback

wtgodbe · 2020-04-01T17:45:46Z

You can double-check things too. Compare the public and private build configurations and confirm the only differences are obvious eg. related to signing.

Doing that now

Attempt to auto-publish vs.redist packages to orchestrated feed

186187f

wtgodbe requested review from dagood, Pilchie and a team March 26, 2020 21:26

wtgodbe commented Mar 26, 2020

View reviewed changes

dagood approved these changes Mar 26, 2020

View reviewed changes

Use non-stable packageVersion for vs.redist

17e1053

dougbu approved these changes Mar 26, 2020

View reviewed changes

Directory.Build.props Outdated Show resolved Hide resolved

Fix typo

9efa5bb

dougbu reviewed Mar 27, 2020

View reviewed changes

Update build/Publish.targets

6466680

- should trigger a build that's more likely to succeed

Pilchie added the area-infrastructure Includes: MSBuild projects/targets, build scripts, CI, Installers and shared framework label Mar 27, 2020

wtgodbe added 2 commits March 27, 2020 12:07

Print what we're trying to publish

3a5b333

Merge branch 'wtgodbe/AutoPublishRedist' of https://github.com/dotnet…

cafdf76

…/aspnetcore into wtgodbe/AutoPublishRedist

JunTaoLuo approved these changes Mar 28, 2020

View reviewed changes

wtgodbe added 2 commits March 31, 2020 09:26

More debug prints

234f97c

Set RedistPackageId in publishing.targets

d44f1b6

wtgodbe commented Mar 31, 2020

View reviewed changes

wtgodbe added 2 commits March 31, 2020 14:14

Reference variable correctly

f0b6d30

Remove unnecessary messages

482d447

wtgodbe marked this pull request as ready for review April 1, 2020 16:30

Merge branch 'release/2.1' into wtgodbe/AutoPublishRedist

5129f7b

dougbu approved these changes Apr 1, 2020

View reviewed changes

Directory.Build.props Outdated Show resolved Hide resolved

build/Publish.targets Outdated Show resolved Hide resolved

Directory.Build.props Outdated Show resolved Hide resolved

build/Publish.targets Show resolved Hide resolved

wtgodbe added 2 commits April 1, 2020 10:35

Respond to feedback

02b837d

Merge branch 'wtgodbe/AutoPublishRedist' of https://github.com/dotnet…

b028dcc

…/aspnetcore into wtgodbe/AutoPublishRedist

wtgodbe merged commit d9c2f74 into release/2.1 Apr 1, 2020

wtgodbe deleted the wtgodbe/AutoPublishRedist branch April 1, 2020 21:45

Attempt to auto-publish vs.redist packages to orchestrated feed #20211

Attempt to auto-publish vs.redist packages to orchestrated feed #20211

Uh oh!

Conversation

wtgodbe commented Mar 26, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

wtgodbe Mar 26, 2020

Choose a reason for hiding this comment

Uh oh!

dougbu Mar 27, 2020

Choose a reason for hiding this comment

Uh oh!

wtgodbe commented Mar 26, 2020

Uh oh!

dagood left a comment

Choose a reason for hiding this comment

Uh oh!

wtgodbe commented Mar 26, 2020

Uh oh!

dougbu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

dougbu commented Mar 26, 2020

Uh oh!

wtgodbe commented Mar 26, 2020

Uh oh!

dougbu commented Mar 26, 2020

Uh oh!

wtgodbe commented Mar 26, 2020

Uh oh!

wtgodbe commented Mar 26, 2020

Uh oh!

wtgodbe commented Mar 26, 2020

Uh oh!

wtgodbe commented Mar 26, 2020

Uh oh!

Uh oh!

dougbu Mar 27, 2020

Choose a reason for hiding this comment

Uh oh!

wtgodbe commented Mar 27, 2020

Uh oh!

wtgodbe commented Mar 27, 2020

Uh oh!

wtgodbe commented Mar 27, 2020

Uh oh!

wtgodbe commented Mar 27, 2020

Uh oh!

wtgodbe commented Mar 27, 2020

Uh oh!

JunTaoLuo left a comment

Choose a reason for hiding this comment

Uh oh!

wtgodbe commented Mar 31, 2020

Uh oh!

wtgodbe commented Mar 31, 2020

Uh oh!

wtgodbe Mar 31, 2020

Choose a reason for hiding this comment

Uh oh!

wtgodbe commented Apr 1, 2020

Uh oh!

wtgodbe commented Apr 1, 2020

Uh oh!

wtgodbe commented Apr 1, 2020

Uh oh!

dougbu commented Apr 1, 2020

Uh oh!

dougbu left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

wtgodbe commented Apr 1, 2020

Uh oh!

wtgodbe commented Apr 1, 2020

wtgodbe commented Mar 26, 2020 •

edited

Loading