Skip to content

[CDRIVER-4644] [CDRIVER-4602] Additional Earthly platforms, functionality, and cleanup #1537

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

vector-of-bool
Copy link
Contributor

@vector-of-bool vector-of-bool commented Feb 21, 2024

This changeset expands the platforms and targets that are executed via Earthly in our CI environment. This is primarily to address CDRIVER-4644 and CDRIVER-4602. In particular, the following additional platforms are now running in Earthly:

  • Ubuntu 18.04
  • Ubuntu 20.04
  • Alpine Linux 3.16, 3.17, and 3.19

Of above, only Ubuntu 18.04 is mentioned directly in CDRIVER-4602, but expanding the reach of tested platforms is trivial and ensures that the CI scripts run correctly across a variety of platforms.

Additionally, another axis of variation was added: GCC versus LLVM/Clang. This supplants the previous ArchLinux tasks that tested both GCC and Clang, whereas previous Earthly-based tasks were only testing with GCC. The GCC/Clang axis is lifted to the build variant level, such that each target operating Linux distribution now has a "GCC" variant and an "LLVM/Clang" variant.

New tools utilities

Earthly has an IMPORT feature that allows one to import an external Earthfile and refer to its targets and commands from an importer. The IMPORT may specify a URL (including a Git repo) or a local directory path. It will read from the Earthfile or build.earth in the imported directory.

A build.earth file has been added to the tools/ subdirectory, and includes non-C-driver-specific utilities for container construction. Each build target environment is based on tools+init-env, which adds the following utility commands that are universally useful:

  • __str, previously str.sh, does simple string manipulation.
  • __install executes the system's package installer "correctly", removing the boilerplate required to perform installs. This script is adapted from the similar script used in libmongocrypt.
  • __boolstr evaluates a command and prints true or false for whether that command exits zero.
  • __alias generates and installs new commands in /usr/local/bin/. Effectively a persistent alias command.
  • __fail prints a message and exits non-zero.
  • The "aliases" __is_alpine, __is_ubuntu, and __is_archlinux are added as simple commands for detecting the host platform.

The init-env base target is carefully constructed to ONLY include things that will very rarely change, since a change at this layer would propagate a cache-invalidating through all derived targets.

Utility Commands

Utility custom Earthly commands have been added: ADD_C_COMPILER, ADD_CXX_COMPILER, ADD_SASL, and ADD_TLS each install the platforms corresponding packages based on the values of --c_compiler, --cxx_compiler, --sasl, and --tls, respectively, for --purpose=build and --purpose=test. These infer the package names required of the container platform. Additionally, the ADD_C_COMPILER and ADD_CXX_COMPILER set the persistent CC and CXX environment variables, respectively.

Each command accepts the same names as used by the corresponding CMake configuration settings, and these commands are therefore used to tweak the container environment for each configuration.

It is expected that additional commands will later be defined for additional configuration axes.

@vector-of-bool vector-of-bool requested a review from kevinAlbs March 1, 2024 18:33
@vector-of-bool vector-of-bool merged commit 9fc8c7f into mongodb:master Apr 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants