Skip to content

Commit 587eb5d

Browse files
committed
ci: use the brand-new GitHub Action to download git-sdk-64-minimal
In our continuous builds, Windows is the odd cookie that requires a complete development environment to be downloaded because there is none installed by default. Side note: technically, there _is_ a development environment present in GitHub Actions' build agents: MSYS2. But it differs from Git for Windows' SDK in subtle points, enough so to prevent Git's test suite from running without failures. Traditionally, we support downloading this environment (which we nicknamed `git-sdk-64-minimal`) via a PowerShell scriptlet that accesses the build artifacts of a dedicated Azure Pipeline (which packages a tiny subset of the full Git for Windows SDK, containing just enough to build Git and run its test suite). This PowerShell script is unfortunately not very robust and sometimes fails due to network issues. Of course, we could add code to detect that situation, wait a little, try again, if it fails again wait a little longer, lather, rinse and repeat. Instead of doing all of this in Git's own `.github/workflows/`, though, let's offload this logic to the brand-new GitHub Action at https://github.com/marketplace/actions/setup-git-for-windows-sdk This Action not only downloads and extracts git-sdk-64-minimal _outside_ the worktree (making it no longer necessary to meddle with `.gitignore`), it also adds the `bash.exe` to the `PATH` and sets the environment variable `MSYSTEM` (an implementation detail that Git's workflow should never have needed to know about). This allows us to convert all those funny PowerShell tasks that wanted to call git-sdk-64-minimal's `bash.exe`: they all are now regular `bash` scriptlets. This finally lets us get rid of the funny quoting and escaping where we had to pay attention not only to quote and escape in the Bash scriptlets properly, but also to add a second level of escaping (with backslashes for double quotes and backticks for dollar signs) so that PowerShell would not do unintended things. Further, this Action uses a fast caching strategy native to GitHub Actions that should accelerate the download across CI runs: git-sdk-64-minimal is usually updated once per 24h, and needs to be cached only once within that period. With this we can drop the home-rolled caching where we try to accelerate the test phase by uploading git-sdk-64-minimal as a workflow artifact after using it to build Git, and then download it as workflow artifact in the test phase. Even better: the `vs-test` job no longer needs to depend on the `windows-build` job. The only reason it depended on it was to ensure that the `git-sdk-64-minimal` workflow artifact was available. Signed-off-by: Johannes Schindelin <[email protected]>
1 parent 773e25a commit 587eb5d

File tree

1 file changed

+19
-84
lines changed

1 file changed

+19
-84
lines changed

.github/workflows/main.yml

Lines changed: 19 additions & 84 deletions
Original file line numberDiff line numberDiff line change
@@ -82,43 +82,18 @@ jobs:
8282
runs-on: windows-latest
8383
steps:
8484
- uses: actions/checkout@v1
85-
- name: download git-sdk-64-minimal
86-
shell: bash
87-
run: |
88-
## Get artifact
89-
urlbase=https://dev.azure.com/git-for-windows/git/_apis/build/builds
90-
id=$(curl "$urlbase?definitions=22&statusFilter=completed&resultFilter=succeeded&\$top=1" |
91-
jq -r ".value[] | .id")
92-
download_url="$(curl "$urlbase/$id/artifacts" |
93-
jq -r '.value[] | select(.name == "git-sdk-64-minimal").resource.downloadUrl')"
94-
curl --connect-timeout 10 --retry 5 --retry-delay 0 --retry-max-time 240 \
95-
-o artifacts.zip "$download_url"
96-
97-
## Unzip and remove the artifact
98-
unzip artifacts.zip
99-
rm artifacts.zip
85+
- uses: git-for-windows/setup-git-for-windows-sdk@v1
10086
- name: build
101-
shell: powershell
87+
shell: bash
10288
env:
10389
HOME: ${{runner.workspace}}
104-
MSYSTEM: MINGW64
10590
NO_PERL: 1
106-
run: |
107-
& .\git-sdk-64-minimal\usr\bin\bash.exe -lc @"
108-
printf '%s\n' /git-sdk-64-minimal/ >>.git/info/exclude
109-
110-
ci/make-test-artifacts.sh artifacts
111-
"@
91+
run: ci/make-test-artifacts.sh artifacts
11292
- name: upload build artifacts
11393
uses: actions/upload-artifact@v1
11494
with:
11595
name: windows-artifacts
11696
path: artifacts
117-
- name: upload git-sdk-64-minimal
118-
uses: actions/upload-artifact@v1
119-
with:
120-
name: git-sdk-64-minimal
121-
path: git-sdk-64-minimal
12297
windows-test:
12398
runs-on: windows-latest
12499
needs: [windows-build]
@@ -136,25 +111,14 @@ jobs:
136111
- name: extract build artifacts
137112
shell: bash
138113
run: tar xf artifacts.tar.gz
139-
- name: download git-sdk-64-minimal
140-
uses: actions/download-artifact@v1
141-
with:
142-
name: git-sdk-64-minimal
143-
path: ${{github.workspace}}/git-sdk-64-minimal/
114+
- uses: git-for-windows/setup-git-for-windows-sdk@v1
144115
- name: test
145-
shell: powershell
146-
run: |
147-
& .\git-sdk-64-minimal\usr\bin\bash.exe -lc @"
148-
# Let Git ignore the SDK
149-
printf '%s\n' /git-sdk-64-minimal/ >>.git/info/exclude
150-
151-
ci/run-test-slice.sh ${{matrix.nr}} 10
152-
"@
116+
shell: bash
117+
run: ci/run-test-slice.sh ${{matrix.nr}} 10
153118
- name: ci/print-test-failures.sh
154119
if: failure()
155-
shell: powershell
156-
run: |
157-
& .\git-sdk-64-minimal\usr\bin\bash.exe -lc ci/print-test-failures.sh
120+
shell: bash
121+
run: ci/print-test-failures.sh
158122
- name: Upload failed tests' directories
159123
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
160124
uses: actions/upload-artifact@v1
@@ -165,27 +129,12 @@ jobs:
165129
needs: ci-config
166130
if: needs.ci-config.outputs.enabled == 'yes'
167131
env:
168-
MSYSTEM: MINGW64
169132
NO_PERL: 1
170133
GIT_CONFIG_PARAMETERS: "'user.name=CI' 'user.email=ci@git'"
171134
runs-on: windows-latest
172135
steps:
173136
- uses: actions/checkout@v1
174-
- name: download git-sdk-64-minimal
175-
shell: bash
176-
run: |
177-
## Get artifact
178-
urlbase=https://dev.azure.com/git-for-windows/git/_apis/build/builds
179-
id=$(curl "$urlbase?definitions=22&statusFilter=completed&resultFilter=succeeded&\$top=1" |
180-
jq -r ".value[] | .id")
181-
download_url="$(curl "$urlbase/$id/artifacts" |
182-
jq -r '.value[] | select(.name == "git-sdk-64-minimal").resource.downloadUrl')"
183-
curl --connect-timeout 10 --retry 5 --retry-delay 0 --retry-max-time 240 \
184-
-o artifacts.zip "$download_url"
185-
186-
## Unzip and remove the artifact
187-
unzip artifacts.zip
188-
rm artifacts.zip
137+
- uses: git-for-windows/setup-git-for-windows-sdk@v1
189138
- name: download vcpkg artifacts
190139
shell: powershell
191140
run: |
@@ -206,38 +155,32 @@ jobs:
206155
shell: bash
207156
run: |
208157
cmake `pwd`/contrib/buildsystems/ -DCMAKE_PREFIX_PATH=`pwd`/compat/vcbuild/vcpkg/installed/x64-windows \
209-
-DMSGFMT_EXE=`pwd`/git-sdk-64-minimal/mingw64/bin/msgfmt.exe -DPERL_TESTS=OFF -DPYTHON_TESTS=OFF -DCURL_NO_CURL_CMAKE=ON
158+
-DMSGFMT_EXE=C:/git-sdk-64-minimal/mingw64/bin/msgfmt.exe -DPERL_TESTS=OFF -DPYTHON_TESTS=OFF -DCURL_NO_CURL_CMAKE=ON
210159
- name: MSBuild
211160
run: msbuild git.sln -property:Configuration=Release -property:Platform=x64 -maxCpuCount:4 -property:PlatformToolset=v142
212161
- name: bundle artifact tar
213-
shell: powershell
162+
shell: bash
214163
env:
215164
MSVC: 1
216165
VCPKG_ROOT: ${{github.workspace}}\compat\vcbuild\vcpkg
217166
run: |
218-
& git-sdk-64-minimal\usr\bin\bash.exe -lc @"
219-
mkdir -p artifacts &&
220-
eval \"`$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts 2>&1 | grep ^tar)\"
221-
"@
167+
mkdir -p artifacts &&
168+
eval "$(make -n artifacts-tar INCLUDE_DLLS_IN_ARTIFACTS=YesPlease ARTIFACTS_DIRECTORY=artifacts 2>&1 | grep ^tar)"
222169
- name: upload build artifacts
223170
uses: actions/upload-artifact@v1
224171
with:
225172
name: vs-artifacts
226173
path: artifacts
227174
vs-test:
228175
runs-on: windows-latest
229-
needs: [vs-build, windows-build]
176+
needs: vs-build
230177
strategy:
231178
fail-fast: false
232179
matrix:
233180
nr: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
234181
steps:
235182
- uses: actions/checkout@v1
236-
- name: download git-sdk-64-minimal
237-
uses: actions/download-artifact@v1
238-
with:
239-
name: git-sdk-64-minimal
240-
path: ${{github.workspace}}/git-sdk-64-minimal/
183+
- uses: git-for-windows/setup-git-for-windows-sdk@v1
241184
- name: download build artifacts
242185
uses: actions/download-artifact@v1
243186
with:
@@ -247,23 +190,15 @@ jobs:
247190
shell: bash
248191
run: tar xf artifacts.tar.gz
249192
- name: test
250-
shell: powershell
193+
shell: bash
251194
env:
252-
MSYSTEM: MINGW64
253195
NO_SVN_TESTS: 1
254196
GIT_TEST_SKIP_REBASE_P: 1
255-
run: |
256-
& .\git-sdk-64-minimal\usr\bin\bash.exe -lc @"
257-
# Let Git ignore the SDK and the test-cache
258-
printf '%s\n' /git-sdk-64-minimal/ /test-cache/ >>.git/info/exclude
259-
260-
ci/run-test-slice.sh ${{matrix.nr}} 10
261-
"@
197+
run: ci/run-test-slice.sh ${{matrix.nr}} 10
262198
- name: ci/print-test-failures.sh
263199
if: failure()
264-
shell: powershell
265-
run: |
266-
& .\git-sdk-64-minimal\usr\bin\bash.exe -lc ci/print-test-failures.sh
200+
shell: bash
201+
run: ci/print-test-failures.sh
267202
- name: Upload failed tests' directories
268203
if: failure() && env.FAILED_TEST_ARTIFACTS != ''
269204
uses: actions/upload-artifact@v1

0 commit comments

Comments
 (0)