Skip to content

Commit 3298900

Browse files
committed
ci: avoid pounding on the poor ci-artifacts container
When this developer tested how the git-sdk-64-minimal artifact could be served to all the GitHub workflow runs that need it, Azure Blobs looked like a pretty good choice: it is reliable, fast and we already use it in Git for Windows to serve components like OpenSSL, cURL, etc It came as an unpleasant surprise just _how many_ times this artifact was downloaded. It exploded the bandwidth to a point where the free tier would no longer be enough, threatening to block other, essential Git for Windows services. Let's switch back to using the Build Artifacts of our trusty Azure Pipeline for the time being. To avoid unnecessary hammering of the Azure Pipeline artifacts, we use the GitHub Action `actions/upload-artifact` in the `windows-build` job and the GitHub Action `actions/download-artifact` in the `windows-test` and `vs-test` jobs (the latter now depends on `windows-build` for that reason, too). Helped-by: Đoàn Trần Công Danh <[email protected]> Signed-off-by: Johannes Schindelin <[email protected]>
1 parent 4267c88 commit 3298900

File tree

1 file changed

+44
-11
lines changed

1 file changed

+44
-11
lines changed

.github/workflows/main.yml

Lines changed: 44 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,19 @@ jobs:
1212
- uses: actions/checkout@v1
1313
- name: download git-sdk-64-minimal
1414
shell: bash
15-
run: a=git-sdk-64-minimal && mkdir -p $a && curl -# https://wingit.blob.core.windows.net/ci-artifacts/$a.tar.xz | tar -C $a -xJf -
15+
run: |
16+
## Get artifact
17+
urlbase=https://dev.azure.com/git-for-windows/git/_apis/build/builds
18+
id=$(curl "$urlbase?definitions=22&statusFilter=completed&resultFilter=succeeded&\$top=1" |
19+
jq -r ".value[] | .id")
20+
download_url="$(curl "$urlbase/$id/artifacts" |
21+
jq -r '.value[] | select(.name == "git-sdk-64-minimal").resource.downloadUrl')"
22+
curl --connect-timeout 10 --retry 5 --retry-delay 0 --retry-max-time 240 \
23+
-o artifacts.zip "$download_url"
24+
25+
## Unzip and remove the artifact
26+
unzip artifacts.zip
27+
rm artifacts.zip
1628
- name: build
1729
shell: powershell
1830
env:
@@ -30,6 +42,11 @@ jobs:
3042
with:
3143
name: windows-artifacts
3244
path: artifacts
45+
- name: upload git-sdk-64-minimal
46+
uses: actions/upload-artifact@v1
47+
with:
48+
name: git-sdk-64-minimal
49+
path: git-sdk-64-minimal
3350
windows-test:
3451
runs-on: windows-latest
3552
needs: [windows-build]
@@ -38,9 +55,6 @@ jobs:
3855
nr: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
3956
steps:
4057
- uses: actions/checkout@v1
41-
- name: download git-sdk-64-minimal
42-
shell: bash
43-
run: a=git-sdk-64-minimal && mkdir -p $a && curl -# https://wingit.blob.core.windows.net/ci-artifacts/$a.tar.xz | tar -C $a -xJf -
4458
- name: download build artifacts
4559
uses: actions/download-artifact@v1
4660
with:
@@ -49,6 +63,11 @@ jobs:
4963
- name: extract build artifacts
5064
shell: bash
5165
run: tar xf artifacts.tar.gz
66+
- name: download git-sdk-64-minimal
67+
uses: actions/download-artifact@v1
68+
with:
69+
name: git-sdk-64-minimal
70+
path: ${{github.workspace}}/git-sdk-64-minimal/
5271
- name: test
5372
shell: powershell
5473
run: |
@@ -79,7 +98,19 @@ jobs:
7998
- uses: actions/checkout@v1
8099
- name: download git-sdk-64-minimal
81100
shell: bash
82-
run: a=git-sdk-64-minimal && mkdir -p $a && curl -# https://wingit.blob.core.windows.net/ci-artifacts/$a.tar.xz | tar -C $a -xJf -
101+
run: |
102+
## Get artifact
103+
urlbase=https://dev.azure.com/git-for-windows/git/_apis/build/builds
104+
id=$(curl "$urlbase?definitions=22&statusFilter=completed&resultFilter=succeeded&\$top=1" |
105+
jq -r ".value[] | .id")
106+
download_url="$(curl "$urlbase/$id/artifacts" |
107+
jq -r '.value[] | select(.name == "git-sdk-64-minimal").resource.downloadUrl')"
108+
curl --connect-timeout 10 --retry 5 --retry-delay 0 --retry-max-time 240 \
109+
-o artifacts.zip "$download_url"
110+
111+
## Unzip and remove the artifact
112+
unzip artifacts.zip
113+
rm artifacts.zip
83114
- name: generate Visual Studio solution
84115
shell: powershell
85116
run: |
@@ -119,15 +150,17 @@ jobs:
119150
path: artifacts
120151
vs-test:
121152
runs-on: windows-latest
122-
needs: [vs-build]
153+
needs: [vs-build, windows-build]
123154
strategy:
124155
matrix:
125156
nr: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
126157
steps:
127158
- uses: actions/checkout@v1
128-
- name: download git-64-portable
129-
shell: bash
130-
run: a=git-64-portable && mkdir -p $a && curl -# https://wingit.blob.core.windows.net/ci-artifacts/$a.tar.xz | tar -C $a -xJf -
159+
- name: download git-sdk-64-minimal
160+
uses: actions/download-artifact@v1
161+
with:
162+
name: git-sdk-64-minimal
163+
path: ${{github.workspace}}/git-sdk-64-minimal/
131164
- name: download build artifacts
132165
uses: actions/download-artifact@v1
133166
with:
@@ -143,9 +176,9 @@ jobs:
143176
NO_SVN_TESTS: 1
144177
GIT_TEST_SKIP_REBASE_P: 1
145178
run: |
146-
& git-64-portable\git-cmd.exe --command=usr\bin\bash.exe -lc @"
179+
& .\git-sdk-64-minimal\usr\bin\bash.exe -lc @"
147180
# Let Git ignore the SDK and the test-cache
148-
printf '%s\n' /git-64-portable/ /test-cache/ >>.git/info/exclude
181+
printf '%s\n' /git-sdk-64-minimal/ /test-cache/ >>.git/info/exclude
149182
150183
cd t &&
151184
PATH=\"`$PWD/helper:`$PATH\" &&

0 commit comments

Comments
 (0)