Skip to content

build-and-deploy: several optimizations, mostly to facilitate arm64 workflows #25

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Feb 8, 2023
Merged

build-and-deploy: several optimizations, mostly to facilitate arm64 workflows #25

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
bbeea79
build-and-deploy: fix builds on arm64
dennisameling Feb 7, 2023
f53d1ab
build-and-deploy: remove initial arm64 hardcoded refs
dennisameling Feb 7, 2023
8f2f906
build-and-deploy: remove cleanup logic on GfW SDK
dennisameling Feb 7, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 17 additions & 11 deletions .github/workflows/build-and-deploy.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -140,8 +140,6 @@ jobs:
flavor: ${{ env.PACKAGE_TO_BUILD == 'mingw-w64-git' && 'build-installers' || 'full' }}
architecture: ${{ env.ARCHITECTURE || 'x86_64' }}
msys: ${{ env.REPO == 'MSYS2-packages' || env.PACKAGE_TO_BUILD == 'git-for-windows-keyring' }}
# We only have to clean up on self-hosted runners
cleanup: ${{ runner.arch == 'ARM64' && true || false }}

- name: Clone ${{ env.REPO }}
shell: bash
Expand Down Expand Up @@ -250,9 +248,20 @@ jobs:
dir="$(cygpath -au artifacts)" &&
mkdir -p "$dir" &&

case "$ARCHITECTURE" in
aarch64)
GIT_EXE="/clangarm64/bin/git.exe"
BUILD_SRC=
;;
*)
GIT_EXE="/mingw64/bin/git.exe"
BUILD_SRC="YesPlease"
;;
esac &&
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we have an x86_64 arm and an i686 arm in this case statement, and a *) arm that errors out?

Also, here would be a fine spot to define MINGW_PREFIX, probably instead of GIT_EXE.

Lastly, given how much time it takes to build the source package, maybe we should consider yet another "ARCHITECTURE" called sources?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused about this one. The build-and-deploy pipeline is currently designed in a way that it uses a single windows-latest runner (x86_64) to build both mingw32 and mingw64 packages in one go. That's why I set up the logic this way.

How would you like to move forward here? Do we want to go for the "one arch, one pipeline" approach like we have in the upcoming git-artifacts pipeline? We'd also use the git-sdk-32 in that case on i686, which has proven to work just fine for mingw-w64-git.

If we can agree on how we want to proceed here, I'll implement the MINGW_PREFIX, "special ARCHITECTURE for sources" and i686/aarch64 in various commits 👍🏼

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The build-and-deploy pipeline is currently designed in a way that it uses a single windows-latest runner (x86_64) to build both mingw32 and mingw64 packages in one go. That's why I set up the logic this way.

Oh whoops. I totally was thinking about the git-artifacts workflow when I said this last night.

In light of that, even the MINGW_PREFIX suggestion does not make sense to me any longer, because there would have to be two such prefixes in the non-ARM64 case.

So I retract my suggestions 😁


{
test -f /usr/bin/git ||
printf '#!/bin/sh\n\nexec /mingw64/bin/git.exe "$@"\n' >/usr/bin/git
printf '#!/bin/sh\n\nexec '$GIT_EXE' "$@"\n' >/usr/bin/git
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we could theoretically use $MINGW_PREFIX/bin/git.exe instead of setting $GIT_EXE in that case.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like that, too.

} &&

MINGW_ARCHS_TO_BUILD=$(
Expand All @@ -265,18 +274,15 @@ jobs:
esac
)

# No need to rebuild i686/x86_64 flavors when building the initial
# arm64 versions of the wintoast/GCM packages
test ebe6c944c573ee4d8900f0e015365468606c9e1e != "$REF" ||
test "mingw32 mingw64 clangarm64" != "$MINGW_ARCHS_TO_BUILD" ||
MINGW_ARCHS_TO_BUILD=clangarm64

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❤️

cd "/usr/src/$REPO/$PACKAGE_TO_BUILD" &&
MAKEFLAGS=-j6 PKGEXT='.pkg.tar.xz' MINGW_ARCH=$MINGW_ARCHS_TO_BUILD $MAKEPKG -s --noconfirm &&
cp *.pkg.tar* "$dir/" &&

MAKEFLAGS=-j6 SRCEXT='.src.tar.gz' MINGW_ARCH=mingw64 $MAKEPKG --allsource &&
cp *.src.tar* "$dir/" &&
if test -n "$BUILD_SRC"
then
MAKEFLAGS=-j6 SRCEXT='.src.tar.gz' MINGW_ARCH=mingw64 $MAKEPKG --allsource &&
cp *.src.tar* "$dir/"
fi &&

# Ensure that the Git worktree is still clean
ls -la &&
Expand Down