Add nanoserver variants #116

tianon · 2016-10-12T20:28:58Z

Build log:

$ docker build 1.7/windows/nanoserver
Sending build context to Docker daemon 3.584 kB
Step 1/9 : FROM microsoft/nanoserver
 ---> e14bc0ecea12
Step 2/9 : SHELL powershell -Command $ErrorActionPreference = 'Stop';
 ---> Using cache
 ---> e44346f1b4f4
Step 3/9 : ENV GOPATH C:\\gopath
 ---> Using cache
 ---> 1971dfb0370e
Step 4/9 : RUN Write-Host 'Updating PATH ...';  $regPath = 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment';            $oldPath = (    Get-ItemProperty                -Path $regPath          -Name PATH      ).path;         Write-Host ('  Before: {0}' -f $oldPath);          $newPath = $env:GOPATH + '\bin;C:\go\bin;' + $oldPath;  Write-Host ('  After:  {0}' -f $newPath);               Set-ItemProperty           -Path $regPath          -Name PATH              -Value $newPath;        Write-Host 'Complete.';
 ---> Running in 7f60e9ea6a5c
Updating PATH ...
 Before: C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\
 After: C:\gopath\bin;C:\go\bin;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\
Complete.
 ---> c783121c618a
Removing intermediate container 7f60e9ea6a5c
Step 5/9 : ENV GOLANG_VERSION 1.7.1
 ---> Running in 909f38eb7885
 ---> 227398d6d8a3
Removing intermediate container 909f38eb7885
Step 6/9 : ENV GOLANG_DOWNLOAD_URL https://golang.org/dl/go$GOLANG_VERSION.windows-amd64.zip
 ---> Running in a1116d1046f9
 ---> 23b31b755435
Removing intermediate container a1116d1046f9
Step 7/9 : ENV GOLANG_DOWNLOAD_SHA256 af2b836bb894672cf4c28df32a2ee3ff560e2b463e1ab44bb99833064ba09e5f
 ---> Running in 30717a793678
 ---> 8674f8a41341
Removing intermediate container 30717a793678
Step 8/9 : RUN Write-Host ('Downloading {0} ...' -f $env:GOLANG_DOWNLOAD_URL);  Invoke-WebRequest -Uri $env:GOLANG_DOWNLOAD_URL -OutFile 'go.zip'; Write-Host ('Verifying sha256 ({0}) ...' -f $env:GOLANG_DOWNLOAD_SHA256);       if ((Get-FileHash go.zip -Algorithm sha256).Hash -ne $env:GOLANG_DOWNLOAD_SHA256) {                Write-Host 'FAILED!';           exit 1;         };              Write-Host 'Expanding ...';     Expand-Archive go.zip -DestinationPath C:\;                Write-Host 'Verifying install ("go version") ...';      go version;             Write-Host 'Removing ...'; Remove-Item go.zip -Force;              Write-Host 'Complete.';
 ---> Running in 759cb3bc667e
Downloading https://golang.org/dl/go1.7.1.windows-amd64.zip ...
Verifying sha256 (af2b836bb894672cf4c28df32a2ee3ff560e2b463e1ab44bb99833064ba09e5f) ...
Expanding ...
Verifying install (go version) ...
go version go1.7.1 windows/amd64
Removing ...
Complete.
 ---> 8c1060d8ea8a
Removing intermediate container 759cb3bc667e
Step 9/9 : WORKDIR $GOPATH
 ---> Running in 30e569aa4a18
 ---> 5bcb73261c5b
Removing intermediate container 30e569aa4a18
Successfully built 5bcb73261c5b

$ docker build 1.6/windows/nanoserver
Sending build context to Docker daemon 3.584 kB
Step 1/9 : FROM microsoft/nanoserver
 ---> e14bc0ecea12
Step 2/9 : SHELL powershell -Command $ErrorActionPreference = 'Stop';
 ---> Using cache
 ---> e44346f1b4f4
Step 3/9 : ENV GOPATH C:\\gopath
 ---> Using cache
 ---> 1971dfb0370e
Step 4/9 : RUN Write-Host 'Updating PATH ...';  $regPath = 'Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Environment';            $oldPath = (    Get-ItemProperty                -Path $regPath          -Name PATH      ).path;         Write-Host ('  Before: {0}' -f $oldPath);          $newPath = $env:GOPATH + '\bin;C:\go\bin;' + $oldPath;  Write-Host ('  After:  {0}' -f $newPath);               Set-ItemProperty           -Path $regPath          -Name PATH              -Value $newPath;        Write-Host 'Complete.';
 ---> Using cache
 ---> c783121c618a
Step 5/9 : ENV GOLANG_VERSION 1.6.3
 ---> Running in 91b72fe396ad
 ---> 56cb42f6de5b
Removing intermediate container 91b72fe396ad
Step 6/9 : ENV GOLANG_DOWNLOAD_URL https://golang.org/dl/go$GOLANG_VERSION.windows-amd64.zip
 ---> Running in edf536cb32a7
 ---> 9012a2f91318
Removing intermediate container edf536cb32a7
Step 7/9 : ENV GOLANG_DOWNLOAD_SHA256 6a18e5ed8b39785338986aecc6a3f36f5c4be286ff52db0ae3bcd2275ab70df0
 ---> Running in cd4325932625
 ---> d89e6d5bb5f9
Removing intermediate container cd4325932625
Step 8/9 : RUN Write-Host ('Downloading {0} ...' -f $env:GOLANG_DOWNLOAD_URL);  Invoke-WebRequest -Uri $env:GOLANG_DOWNLOAD_URL -OutFile 'go.zip'; Write-Host ('Verifying sha256 ({0}) ...' -f $env:GOLANG_DOWNLOAD_SHA256);       if ((Get-FileHash go.zip -Algorithm sha256).Hash -ne $env:GOLANG_DOWNLOAD_SHA256) {                Write-Host 'FAILED!';           exit 1;         };              Write-Host 'Expanding ...';     Expand-Archive go.zip -DestinationPath C:\;                Write-Host 'Verifying install ("go version") ...';      go version;             Write-Host 'Removing ...'; Remove-Item go.zip -Force;              Write-Host 'Complete.';
 ---> Running in 156a886a5ba3
Downloading https://golang.org/dl/go1.6.3.windows-amd64.zip ...
Verifying sha256 (6a18e5ed8b39785338986aecc6a3f36f5c4be286ff52db0ae3bcd2275ab70df0) ...
Expanding ...
Verifying install (go version) ...
go version go1.6.3 windows/amd64
Removing ...
Complete.
 ---> 342fc585ebd1
Removing intermediate container 156a886a5ba3
Step 9/9 : WORKDIR $GOPATH
 ---> Running in a9ba16943437
 ---> 79a8be691d8d
Removing intermediate container a9ba16943437
Successfully built 79a8be691d8d

Go 1.7 is ~1.12GB total, Go 1.6 is ~1.19GB total.

tianon · 2016-10-12T20:29:58Z

1.6/windows/nanoserver/Dockerfile

+		-Path $regPath \
+		-Name PATH \
+		-Value $newPath; \
+	Write-Host 'Complete.';


I couldn't get any incarnation of [Environment]::SetEnvironmentVariable(...) to work on Nano Server, hence this new manual registry manipulation. 😞

@tianon setx also works (it's a cmd.exe command, but should be in powershell too): https://github.com/dotnet/dotnet-docker/blob/master/1.0.0-preview2/nanoserver/Dockerfile#L13

Heh OK, I'll try to swallow my PowerShell purism in favor of the pragmatic significantly-less-lines approach of shelling out to setx 😇

tianon · 2016-10-12T20:30:32Z

1.6/windows/nanoserver/Dockerfile

+ENV GOLANG_DOWNLOAD_SHA256 6a18e5ed8b39785338986aecc6a3f36f5c4be286ff52db0ae3bcd2275ab70df0
+
+RUN Write-Host ('Downloading {0} ...' -f $env:GOLANG_DOWNLOAD_URL); \
+	Invoke-WebRequest -Uri $env:GOLANG_DOWNLOAD_URL -OutFile 'go.zip'; \


Invoke-WebRequest supposedly buffers in memory, but it's the only option available in Nano Server. 😞

Have a look at the Docker PowerShell provider. They have added a BitsOnNano.exe

https://www.powershellgallery.com/packages/DockerMsftProvider/1.0.0.1 (and open the FileList)

😄

The source for that is here: https://github.com/OneGet/MicrosoftDockerProvider/tree/developer

Interesting!

I was really hoping to use the same .NET methods we use in Server Core (New-Object System.Net.WebClient) since they offer IMO the best balance of performance and usability (based on my research, not on real-world testing), but of the three main "download files" methods in PowerShell core, Invoke-WebRequest was the only one available in stock Nano, so I was just having a bit of a sad moment.

I haven't actually verified the claim that I've read on the internet somewhere that Invoke-WebRequest buffers in memory (but it's online!! it has to be true 😅), and IMO it's probably not worth the overhead of installing an unrelated module (even if we remove it afterwards) to avoid that. I think our build server can handle it even if this does buffer in-memory. 😄 ❤️

@tianon Invoke-WebRequest used to be slow, but I think it's ok now when used with -UseBasicParsing. I don't know about the buffering behavior though.

MicrosoftDocs/Virtualization-Documentation#360 (comment)

Oh I just realised that PowerShell is on GitHub, and found a similar issue PowerShell/PowerShell#2138
But does this impact running in a container without visible progress bar?

Yes, narrowed it down. Using something like

RUN $ProgressPreference = 'SilentlyContinue'; ` Write-Host ('Downloading {0} with Invoke-WebRequest SilentlyContinue ...' -f $env:GOLANG_DOWNLOAD_URL); ` Measure-Command { Invoke-WebRequest -Uri $env:GOLANG_DOWNLOAD_URL -OutFile 'go.zip' }

works fast!

Nice! I guess we should add $ProgressPreference = 'SilentlyContinue'; to our SHELL line along with $StopActionPreference given that the issue you linked to mentions other commands use $ProgressPreference too like Expand-Archive, right?

ie PowerShell/PowerShell#2138 (comment)

Yes, I also thought adding it to SHELL. And yes this should help for Expand-Archive as well. :-)

Ya, after finding this out I started adding SilentlyContinue to pretty much every system I use :-)

friism

❤️ ❤️ ❤️

ping @StefanScherer

friism · 2016-10-12T20:48:24Z

1.6/windows/nanoserver/Dockerfile

+		-Path $regPath \
+		-Name PATH \
+		-Value $newPath; \
+	Write-Host 'Complete.';


@tianon setx also works (it's a cmd.exe command, but should be in powershell too): https://github.com/dotnet/dotnet-docker/blob/master/1.0.0-preview2/nanoserver/Dockerfile#L13

StefanScherer · 2016-10-12T20:59:17Z

That is awesome! So CI is able to build nanoserver images, then? Are you buildling against the 10.0.14393.321 images tagged a few hours ago?

tianon · 2016-10-14T18:11:47Z

@StefanScherer we got Server 2016 RTM and it was able to run Nano images, so we're equipped to build them officially now (and have a separate server for testing/development work), but we don't have CI that can do either Nano or Server Core yet

StefanScherer · 2016-10-14T18:31:55Z

@tianon Same for me, building locally in VM's, but waiting for AppVeyor to have Docker support :-)

…nts for simpler cross-variant diffing)

tianon · 2016-10-14T21:30:12Z

Alright, pull request is updated to be simpler and have a smaller diff nanoserver -> windowsservercore.

I also watched memory usage while two of these Invoke-WebRequest lines were running, and it stayed completely constant (it actually went up more for extracting the zips than for downloading them, as one might expect). 👍

tianon · 2016-10-14T21:31:42Z

For easier review:

diff --git a/1.7/windows/windowsservercore/Dockerfile b/1.7/windows/nanoserver/Dockerfile
index c1b1bcd..6d6a4c8 100644
--- a/1.7/windows/windowsservercore/Dockerfile
+++ b/1.7/windows/nanoserver/Dockerfile
@@ -1,55 +1,9 @@
-FROM microsoft/windowsservercore
+FROM microsoft/nanoserver

 SHELL ["powershell", "-Command", "$ErrorActionPreference = 'Stop';"]

-# install Git (especially for "go get")
-ENV GIT_VERSION 2.9.2
-ENV GIT_TAG v${GIT_VERSION}.windows.1
-ENV GIT_DOWNLOAD_URL https://github.com/git-for-windows/git/releases/download/${GIT_TAG}/Git-${GIT_VERSION}-64-bit.exe
-ENV GIT_DOWNLOAD_SHA256 006d971bcbe73cc8d841a100a4eb20d22e135142bf5b0f2120722fd420e166e5
-# steps inspired by "chcolateyInstall.ps1" from "git.install" (https://chocolatey.org/packages/git.install)
-RUN Write-Host ('Downloading {0} ...' -f $env:GIT_DOWNLOAD_URL); \
-   Invoke-WebRequest -Uri $env:GIT_DOWNLOAD_URL -OutFile 'git.exe'; \
-   \
-   Write-Host ('Verifying sha256 ({0}) ...' -f $env:GIT_DOWNLOAD_SHA256); \
-   if ((Get-FileHash git.exe -Algorithm sha256).Hash -ne $env:GIT_DOWNLOAD_SHA256) { \
-       Write-Host 'FAILED!'; \
-       exit 1; \
-   }; \
-   \
-   Write-Host 'Installing ...'; \
-   Start-Process \
-       -Wait \
-       -FilePath ./git.exe \
-# http://www.jrsoftware.org/ishelp/topic_setupcmdline.htm
-       -ArgumentList @( \
-           '/VERYSILENT', \
-           '/NORESTART', \
-           '/NOCANCEL', \
-           '/SP-', \
-           '/SUPPRESSMSGBOXES', \
-           \
-# https://github.com/git-for-windows/build-extra/blob/353f965e0e2af3e8c993930796975f9ce512c028/installer/install.iss#L87-L96
-           '/COMPONENTS=assoc_sh', \
-           \
-# set "/DIR" so we can set "PATH" afterwards
-# see https://disqus.com/home/discussion/chocolatey/chocolatey_gallery_git_install_1710/#comment-2834659433 for why we don't use "/LOADINF=..." to let the installer set PATH
-           '/DIR=C:\git' \
-       ); \
-   \
-   Write-Host 'Updating PATH ...'; \
-   $env:PATH = 'C:\git\bin;C:\git\mingw64\bin;C:\git\usr\bin;' + $env:PATH; \
-   [Environment]::SetEnvironmentVariable('PATH', $env:PATH, [EnvironmentVariableTarget]::Machine); \
-   \
-   Write-Host 'Verifying install ...'; \
-   Write-Host '  git --version'; git --version; \
-   Write-Host '  bash --version'; bash --version; \
-   Write-Host '  curl --version'; curl.exe --version; \
-   \
-   Write-Host 'Removing installer ...'; \
-   Remove-Item git.exe -Force; \
-   \
-   Write-Host 'Complete.';
+# no Git installed (intentionally)
+#  -- Nano Server is "Windows Slim"

 # ideally, this would be C:\go to match Linux a bit closer, but C:\go is the recommended install path for Go itself on Windows
 ENV GOPATH C:\\gopath
@@ -57,7 +11,8 @@ ENV GOPATH C:\\gopath
 # PATH isn't actually set in the Docker image, so we have to set it from within the container
 RUN $newPath = ('{0}\bin;C:\go\bin;{1}' -f $env:GOPATH, $env:PATH); \
    Write-Host ('Updating PATH: {0}' -f $newPath); \
-   [Environment]::SetEnvironmentVariable('PATH', $newPath, [EnvironmentVariableTarget]::Machine);
+# Nano Server does not have "[Environment]::SetEnvironmentVariable()"
+   setx /M PATH $newPath;
 # doing this first to share cache across versions more aggressively

 ENV GOLANG_VERSION 1.7.1

tianon · 2016-10-14T21:34:06Z

(I also still have all four build logs open -- all successful -- but I don't think pasting them here will add much value since they're pretty similar to the previous ones I already pasted. 😄)

- `celery`: 4.0.0rc5 - `elasticsearch`: refactor template (docker-library/elasticsearch#124) - `golang`: add `nanoserver` variants (docker-library/golang#116) - `java`: debian 9~b140-1 - `kibana`: fix `server.host` in 5+ (docker-library/kibana#60) - `mariadb`: 5.5.53+maria-1~wheezy - `openjdk`: debian 9~b140-1 - `php`: 5.6.27; template refactor for better pre-releases support (docker-library/php#314) - `ruby`: bundler 1.13.5

tianon commented Oct 12, 2016

View reviewed changes

friism reviewed Oct 12, 2016

View reviewed changes

Add nanoserver variants (and adjust bits of "windowsservercore" varia…

cba64e4

…nts for simpler cross-variant diffing)

tianon force-pushed the nanoserver branch from 70b09d4 to cba64e4 Compare October 14, 2016 21:13

yosifkit approved these changes Oct 14, 2016

View reviewed changes

tianon merged commit aa5a959 into docker-library:master Oct 14, 2016

tianon deleted the nanoserver branch October 14, 2016 22:01

tianon mentioned this pull request Oct 18, 2016

Update docker-library images docker-library/official-images#2263

Merged

tianon mentioned this pull request Dec 1, 2016

Add "$ProgressPreference = 'SilentlyContinue';" to Windows-based variants #124

Merged

Add nanoserver variants #116

Add nanoserver variants #116

Uh oh!

Conversation

tianon commented Oct 12, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

friism left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

StefanScherer commented Oct 12, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tianon commented Oct 14, 2016

Uh oh!

StefanScherer commented Oct 14, 2016

Uh oh!

tianon commented Oct 14, 2016

Uh oh!

tianon commented Oct 14, 2016

Uh oh!

tianon commented Oct 14, 2016

Uh oh!

Uh oh!

StefanScherer commented Oct 12, 2016 •

edited

Loading