Skip to content

PHPLIB-1098: Update evergreen build hosts #1442

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Jun 28, 2023
Merged
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
77 changes: 61 additions & 16 deletions .evergreen/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -361,7 +361,7 @@ functions:
params:
script: |
${PREPARE_SHELL}
MONGODB_VERSION=${MONGODB_VERSION} TOPOLOGY=${TOPOLOGY} AUTH=${AUTH} SSL=${SSL} STORAGE_ENGINE=${STORAGE_ENGINE} LOAD_BALANCER=${LOAD_BALANCER} REQUIRE_API_VERSION=${REQUIRE_API_VERSION} ORCHESTRATION_FILE=${ORCHESTRATION_FILE} sh ${DRIVERS_TOOLS}/.evergreen/run-orchestration.sh
SKIP_LEGACY_SHELL=${SKIP_LEGACY_SHELL} MONGODB_VERSION=${MONGODB_VERSION} TOPOLOGY=${TOPOLOGY} AUTH=${AUTH} SSL=${SSL} STORAGE_ENGINE=${STORAGE_ENGINE} LOAD_BALANCER=${LOAD_BALANCER} REQUIRE_API_VERSION=${REQUIRE_API_VERSION} ORCHESTRATION_FILE=${ORCHESTRATION_FILE} sh ${DRIVERS_TOOLS}/.evergreen/run-orchestration.sh
# run-orchestration generates expansion file with MONGODB_URI and CRYPT_SHARED_LIB_PATH
- command: expansions.update
params:
Expand Down Expand Up @@ -1124,27 +1124,58 @@ axes:
- id: os
display_name: OS
values:
# Debian
- id: debian11
display_name: "Debian 11"
run_on: debian11
run_on: debian11-small
- id: debian10
display_name: "Debian 10"
run_on: debian10
run_on: debian10-small
- id: debian92
display_name: "Debian 9.2"
run_on: debian92
- id: rhel70
display_name: "RHEL 7.0"
run_on: rhel70
run_on: debian92-small

# RHEL
- id: rhel90
display_name: "RHEL 9.0"
run_on: rhel90-small
variables:
SKIP_LEGACY_SHELL: "true"
# TODO: RHEL 8.x is not working yet
# - id: rhel82-arm64
# display_name: "RHEL 8.2 ARM64"
# run_on: rhel82-arm64-small
# - id: rhel80
# display_name: "RHEL 8.0"
# run_on: rhel80-small
- id: rhel76
display_name: "RHEL 7.6"
run_on: rhel76-small
- id: rhel71-power8
display_name: "RHEL 7.1 Power 8"
run_on: rhel71-power8-build
- id: rhel72-zseries
display_name: "RHEL 7.2 zSeries"
run_on: rhel72-zseries-build
- id: ubuntu1804-arm64
display_name: "Ubuntu 18.04 ARM64"
run_on: ubuntu1804-arm64-test

# Ubuntu LTS
- id: ubuntu2204
display_name: "Ubuntu 22.04 x64"
run_on: ubuntu2204-small
variables:
SKIP_LEGACY_SHELL: "true"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is SKIP_LEGACY_SHELL here in any way related to mongodb/mongo-php-library#1035 and PHPLIB-1028?

I only found SKIP_LEGACY_SHELL referenced in download-mongodb.sh. In that script, it is only used in a test for -z "${SKIP_LEGACY_SHELL:-}" (i.e. branch happens when the variable is zero-length or undefined). So in that case, defining this string as anything is considered a "true" value to avoid the conditional download of the legacy shell. That seems a bit odd to me, but I suppose it's just because we tend to do explicit checks for "yes" values in our own scripts.

Why do we only do this for the Ubuntu 22.04 platforms? If the legacy shell isn't necessary at all, perhaps we should be setting SKIP_LEGACY_SHELL for all environments.

I see download-mongodb.sh is only called in our config for the getdata task (only executed manually for debugging). I suppose the only other place it is called is via the bootstrap mongo-orchestration task through run-orchestration.sh.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is SKIP_LEGACY_SHELL here in any way related to mongodb/mongo-php-library#1035 and PHPLIB-1028?

That is correct. The corresponding DRIVERS ticket was blocked on an issue with the AWS tests, which I think is resolved that. I'll follow up on that separately.

I only found SKIP_LEGACY_SHELL referenced in download-mongodb.sh. In that script, it is only used in a test for -z "${SKIP_LEGACY_SHELL:-}" (i.e. branch happens when the variable is zero-length or undefined). So in that case, defining this string as anything is considered a "true" value to avoid the conditional download of the legacy shell. That seems a bit odd to me, but I suppose it's just because we tend to do explicit checks for "yes" values in our own scripts.

I believe I modelled this after SKIP_CRYPT_SHARED, which uses the same system. Similarly, SERVERLESS_SKIP_CRYPT also requires any value different from OFF to be defined. This could be changed, but more on that below.

Why do we only do this for the Ubuntu 22.04 platforms? If the legacy shell isn't necessary at all, perhaps we should be setting SKIP_LEGACY_SHELL for all environments.

For the time being, I've only disabled this for platforms that don't have a legacy shell. However, after double-checking, neither drivers-evergreen-tools nor our own evergreen scripts make use of the legacy shell, so I've changed this to skip installing the legacy shell on all platforms.

I see download-mongodb.sh is only called in our config for the getdata task (only executed manually for debugging). I suppose the only other place it is called is via the bootstrap mongo-orchestration task through run-orchestration.sh.

That is correct - run-orchestration.sh downloads the requested MongoDB version through download-mongodb.sh and then starts the requested topology.

- id: ubuntu2204-arm64
display_name: "Ubuntu 22.04 ARM64"
run_on: ubuntu2204-arm64-small
variables:
SKIP_LEGACY_SHELL: "true"
- id: ubuntu2004
display_name: "Ubuntu 20.04 x64"
run_on: ubuntu2004-small
- id: ubuntu2004-arm64
display_name: "Ubuntu 20.04 ARM64"
run_on: ubuntu2004-arm64-small

# Pending installation of PHP toolchain on macOS hosts (see: PHPC-869)
# - id: macos-1014
# display_name: "Mac OS 10.14"
Expand Down Expand Up @@ -1184,15 +1215,29 @@ axes:


buildvariants:
# Test all PHP versions with latest-stable MongoDB on Debian 11 and RHEL 7.0
# Test all PHP versions with latest-stable MongoDB on Debian 11, Debian 10, RHEL 8.0, Ubuntu 20.04
- matrix_name: "test-php-versions"
matrix_spec: { "os": ["rhel70", "debian11"], "mongodb-edge-versions": "latest-stable", "php-versions": "*" }
matrix_spec:
os:
- debian11
- debian10
- rhel90
# TODO: RHEL 8.x is not working yet
# - rhel82-arm64
# - rhel80
- rhel76
- ubuntu2204-arm64
- ubuntu2204
- ubuntu2004-arm64
- ubuntu2004
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it redundant to test PHP 8.1+ on both ubuntu2204 and ubuntu2004 (x64 and arm64)?

To make more efficient use of Evergreen, perhaps we can split this around the version boundary, similar to what we do with debian92 and debian11 in the test-mongodb-versions matrix.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this involves different versions of OpenSSL I found it prudent to test the full range. Also, only testing the old versions would have us continuously update the list of excluded PHP versions on Ubuntu 20.04 as new versions are added. I'm not that concerned about the load on Evergreen on checks running on our branches, but I think it would make sense to limit the tasks we run when testing pull request. I don't think we need to test the full complement of PHP versions on all operating systems for that.

Potentially, we can re-address this when we refactor our build matrices in PHPC-2201.

mongodb-edge-versions: latest-stable
php-versions: "*"
display_name: "${os}, ${mongodb-edge-versions}, ${php-versions}"
exclude_spec:
# Exclude "latest-stable" PHP version for Debian 11 (see: test-mongodb-versions matrix)
- { "os": "debian11", "mongodb-edge-versions": "*", "php-versions": "8.2" }
# PHP 8.1+ is not available on rhel70
- { "os": "rhel70", "mongodb-edge-versions": "*", "php-versions": ["8.1", "8.2"] }
# Exclude PHP versions older than 8.1 on RHEL 9 and Ubuntu 22.04 (OpenSSL 3 is only supported on PHP 8.1+)
- { "os": ["rhel90", "ubuntu2204-arm64", "ubuntu2204"], "mongodb-edge-versions": "*", "php-versions": ["7.2", "7.3", "7.4", "8.0"] }
tasks:
- name: "test-standalone-ssl"
- name: "test-replicaset-auth"
Expand All @@ -1214,9 +1259,9 @@ buildvariants:
- name: "test-replicaset-auth"
- name: "test-sharded"

# Test ARM64, Power8, and zSeries architectures with MongoDB 4.4
# Test RHEL Power8 and zSeries architectures with MongoDB 4.4
- matrix_name: "test-alt-archs"
matrix_spec: { "os": ["rhel71-power8", "rhel72-zseries", "ubuntu1804-arm64"], "mongodb-versions": "4.4", "php-edge-versions": "oldest-supported" }
matrix_spec: { "os": ["rhel71-power8", "rhel72-zseries"], "mongodb-versions": "4.4", "php-edge-versions": "oldest-supported" }
display_name: "${os}, ${mongodb-versions}, ${php-edge-versions}"
tasks:
- name: "test-standalone-ssl"
Expand Down