Can't publish module to local repository since v2.1.3

[kborowinski]

Steps to reproduce

Register local repository:

$repo = @{
Name='Test'
SourceLocation='D:\Test'
PublishLocation='D:\Test'
InstallationPolicy='trusted'
}
Register-PSRepository @repo

Try to publish any module to local repository:

Publish-Module -Repository Test -Verbose -Name Pester

Expected behavior

Module is published successfully on PowerShellGet v2.1.2

Actual behavior

Publish-Module hangs indefinitely on PowerShellGet v2.1.3

Environment data

> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14393.2879
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.2879
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

> Get-Module -ListAvailable PowerShellGet,PackageManagement

    Directory: C:\Program Files\WindowsPowerShell\Modules

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     1.3.2      PackageManagement                   {Find-Package, Get-Package, Get-PackageProvider, Get-PackageSource...}
Script     2.1.2      PowerShellGet                       {Find-Command, Find-DSCResource, Find-Module, Find-RoleCapability...}

> Get-PackageProvider -ListAvailable

Name                     Version          DynamicOptions
----                     -------          --------------
msi                      3.0.0.0          AdditionalArguments
msu                      3.0.0.0
nuget                    2.8.5.208
NuGet                    3.0.0.1          Destination, ExcludeVersion, Scope, SkipDependencies, Headers, FilterOnTag, Contains, AllowPrereleaseVer... PowerShellGet            2.1.2.0          PackageManagementProvider, Type, Scope, AllowClobber, SkipPublisherCheck, InstallUpdate, NoPathUpdate, A...
Programs                 3.0.0.0          IncludeWindowsInstaller, IncludeSystemComponent

[edyoung]

That's quite odd. If you explicitly run the previous build (import-module powershellget -requiredversion ...) does the issue reproduce?

[kborowinski]

When I import previous build explicitly it works as expected. The problem happens with new build. I did also check it on second machine with the same result. And it does not matter if the repository is local or UNC path.

[kborowinski]

OK, it's weird. I've tried to publish other popular module - Plaster. And it works with both builds. So I did little test and reinstalled latest Pester module from PSGallery (4.8.1) but it didn't help and I can't publish it with v2.1.3.

Edit:
It looks like it is module dependent. I was lucky with Plaster. I can't publish any of the following modules with v2.1.3 to the local repository (it works with previous build though):
ImportExcel
dbatools
PoshRsJob
psake
PSScriptAnalyzer

[edyoung]

Thanks! I can reproduce the issue. Nuget.exe is not terminating when we run it to generate the package. So far I'm not sure why, the same command-line completes just fine when run interactively.

[edyoung]

The issue is, when building some packages nuget writes enough data to standard out to fill the buffer, then blocks. But the calling script doesn't read anything from the output until it nuget has exited. It only affects some packages because different packages produce different amounts of output from nuget/dotnet.

[sandersaares]

2.1.3 appears to be installed in Azure Pipelines hosted agents, so I am running into this there. Maybe you can drop a word to whoever maintains that that this fix could be welcome?

I am not even sure how to upgrade, Update-Module does not appear to install a newer version there for some reason.

[avishnyakov]

@edyoung nice finding. How to work around it?
Having my module publishing stuck at "VERBOSE: Performing the operation "Publish-Module" on target ....", found this thread and how scratching my head.

The issue is, when building some packages nuget writes enough data to standard out to fill the buffer, then blocks. But the calling script doesn't read anything from the output until it nuget has exited. It only affects some packages because different packages produce different amounts of output from nuget/dotnet.

[avishnyakov]

In my case, I have sub-folders for my customer PowerShell module.
NuGet stuck, indeed. The dotnet.exe process stuck in the process explorer. Once killed, here is what we see in the verbose output.

Trying to find a way to trick nuget/dotnet. Placing everything into /tools sub-folder didn't work.
Now I wonder how come other

ERROR: Failed to generate the compressed file for module 'dotnet cli failed to pack M
icrosoft (R) Build Engine version 16.4.0+e901037fe for .NET Core
Copyright (C) Microsoft Corporation. All rights reserved.

  Restore completed in 275.36 ms for C:\Users\user\AppData\Local\Temp\f44
2cac3-564d-4afe-b84d-6cdc6a36d902\Temp.csproj.
  Temp -> C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b84d-6cdc6a
36d902\bin\Debug\netcoreapp2.0\NotUsed.dll
  Successfully created package 'C:\Users\user\AppData\Local\Temp\84405364
2\InvokeMyModule\InvokeMyModule.0.2.2002.1129.nupkg'.
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tas
ks.Pack.targets(198,5): warning NU5110: The script file 'InvokeMyModule.ps1' is outside 
the 'tools' folder and hence will not be executed during installation of this package
. Move it into the 'tools' folder. [C:\Users\user\AppData\Local\Temp\f442
cac3-564d-4afe-b84d-6cdc6a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tas
ks.Pack.targets(198,5): warning NU5111: The script file 'InvokeMyModule.ps1' is not reco
gnized by NuGet and hence will not be executed during installation of this package. R
ename it to install.ps1, uninstall.ps1 or init.ps1 and place it directly under 'tools
'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b84d-6cdc6a36d902
\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tas
ks.Pack.targets(198,5): warning NU5111: The script file 'tools\deploy.ps1' is not rec
ognized by NuGet and hence will not be executed during installation of this package. 
Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it directly under 'tool
s'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b84d-6cdc6a36d90
2\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tas
ks.Pack.targets(198,5): warning NU5111: The script file 'tools\components\CloudCompon
ent.ps1' is not recognized by NuGet and hence will not be executed during installatio
n of this package. Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it d
irectly under 'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4a
fe-b84d-6cdc6a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tas
ks.Pack.targets(198,5): warning NU5111: The script file 'tools\core\BaseClass.ps1' is
 not recognized by NuGet and hence will not be executed during installation of this p
ackage. Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it directly und
er 'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b84d-6cd
c6a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tas
ks.Pack.targets(198,5): warning NU5111: The script file 'tools\core\CliUtils.ps1' is 
not recognized by NuGet and hence will not be executed during installation of this pa
ckage. Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it directly unde
r 'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b84d-6cdc
6a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tas
ks.Pack.targets(198,5): warning NU5111: The script file 'tools\core\DeploymentContext
.ps1' is not recognized by NuGet and hence will not be executed during installation o
f this package. Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it dire
ctly under 'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-
b84d-6cdc6a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Tas
ks.Pack.targets(198,5): warning NU5111: The script file 'tools\core\DeploymentStage.p
s1' is not recognized by NuGet and hence will not be executed during installation of 
this package. Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it direct
ly under 'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b8
4d-6cdc6a36d902\Temp.csproj]
'.
At C:\program files\powershell\6\Modules\PowerShellGet\PSModule.psm1:10810 char:17
+ ...             Publish-PSArtifactUtility @PublishPSArtifactUtility_Param ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
At C:\Users\user\source\repos\my-module\platform-MyModule-cli\.build-sha
red.ps1:157 char:1
+ task BuildModule {
+ ~~~~~~~~~~~~~~~~~~
At C:\Users\user\source\repos\my-module\platform-MyModule-cli\.build-sha
red.ps1:285 char:1
+ task DefaultBuild UnitTest,
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
At C:\Users\user\source\repos\my-module\platform-MyModule-cli\.build-sha
red.ps1:294 char:1
+ task . DefaultBuild
+ ~~~~~~~~~~~~~~~~~~~
Build FAILED. 8 tasks, 1 errors, 0 warnings 00:00:33.0177652
pwsh : Publish-PSArtifactUtility : Failed to generate the compressed file for 
module 'dotnet cli failed to pack Microsoft (R) Build Engine version 
16.4.0+e901037fe for .NET Core
At line:6 char:1
+ pwsh -c invoke-build
+ ~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Publish-PSArtif...e for .NET Core:Stri 
   ng) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError
 
Copyright (C) Microsoft Corporation. All rights reserved.
  Restore completed in 275.36 ms for C:\Users\user\AppData\Local\Temp\f4
42cac3-564d-4afe-b84d-6cdc6a36d902\Temp.csproj.
  Temp -> C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b84d-6cdc6
a36d902\bin\Debug\netcoreapp2.0\NotUsed.dll
  Successfully created package 'C:\Users\user\AppData\Local\Temp\8440536
42\InvokeMyModule\InvokeMyModule.0.2.2002.1129.nupkg'.
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Ta
sks.Pack.targets(198,5): warning NU5110: The script file 'InvokeMyModule.ps1' is 
outside the 'tools' folder and hence will not be executed during installation of 
this package.
 Move it into the 'tools' folder. [C:\Users\user\AppData\Local\Temp\f442
cac3-564d-4afe-b84d-6cdc6a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Ta
sks.Pack.targets(198,5): warning NU5111: The script file 'InvokeMyModule.ps1' is not 
recognized by NuGet and hence will not be executed during installation of this 
package. Re
name it to install.ps1, uninstall.ps1 or init.ps1 and place it directly under 
'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b84d-6cdc6
a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Ta
sks.Pack.targets(198,5): warning NU5111: The script file 'tools\deploy.ps1' is not 
recognized by NuGet and hence will not be executed during installation of this 
package. R
ename it to install.ps1, uninstall.ps1 or init.ps1 and place it directly under 
'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b84d-6cdc6
a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Ta
sks.Pack.targets(198,5): warning NU5111: The script file 
'tools\components\CloudComponent.ps1' is not recognized by NuGet and hence will not 
be executed during installation
 of this package. Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it 
directly under 'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-
4afe-b84d-6cdc6a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Ta
sks.Pack.targets(198,5): warning NU5111: The script file 'tools\core\BaseClass.ps1' 
is not recognized by NuGet and hence will not be executed during installation of 
this pa
ckage. Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it directly 
under 'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b84d
-6cdc6a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Ta
sks.Pack.targets(198,5): warning NU5111: The script file 'tools\core\CliUtils.ps1' 
is not recognized by NuGet and hence will not be executed during installation of 
this pac
kage. Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it directly 
under 'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-4afe-b84d
-6cdc6a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Ta
sks.Pack.targets(198,5): warning NU5111: The script file 
'tools\core\DeploymentContext.ps1' is not recognized by NuGet and hence will not be 
executed during installation of
 this package. Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it 
directly under 'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-
4afe-b84d-6cdc6a36d902\Temp.csproj]
C:\Program Files\dotnet\sdk\3.1.101\Sdks\NuGet.Build.Tasks.Pack\build\NuGet.Build.Ta
sks.Pack.targets(198,5): warning NU5111: The script file 
'tools\core\DeploymentStage.ps1' is not recognized by NuGet and hence will not be 
executed during installation of t
his package. Rename it to install.ps1, uninstall.ps1 or init.ps1 and place it 
directly under 'tools'. [C:\Users\user\AppData\Local\Temp\f442cac3-564d-
4afe-b84d-6cdc6a36d902\Temp.csproj]
'.
At C:\program files\powershell\6\Modules\PowerShellGet\PSModule.psm1:10810 char:17
+ ...             Publish-PSArtifactUtility @PublishPSArtifactUtility_Param ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidOperation: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : FailedToCreateCompressedModule,Publish-PSArtifactUtility

[avishnyakov]

@edyoung I might be wrong but maybe we might need to add something like for the PR https://github.com/PowerShell/PowerShellGet/pull/479 while generating $CsprojContent?

<NoWarn>NU5110;NU5111</NoWarn>

https://github.com/PowerShell/PowerShellGet/pull/479/files#diff-8a5643a86665a53bd2054068d6f33fe5R60

Feels that we might need to suppress output warnings by dotnet pack for some warning. These might be exactly the warning which go to the output and locking dotnet.exe.

Looking into how the output is read, it indeed can get full and async read might help. Similar challenge mentioned on SO:

• https://stackoverflow.com/a/7608823

[avishnyakov]

Same issue under linux container, azuresdk/azure-powershell-core with dotnet-sdk 3.1