Skip to content

PHPLIB-950: Run legacy CSFLE tests on serverless #988

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 6, 2022
Merged

PHPLIB-950: Run legacy CSFLE tests on serverless #988

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
5aae9ef
Download mongocryptd and crypt_shared for serverless tests
alcaeus Oct 5, 2022
6656199
Export FLE secrets to enable testing
alcaeus Oct 5, 2022
42a8014
Run KMS servers when testing against serverless
alcaeus Oct 5, 2022
8c0111c
PHPLIB-950: Run legacy CSFLE tests on serverless
alcaeus Oct 5, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 63 additions & 0 deletions .evergreen/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -212,6 +212,55 @@ functions:
- command: expansions.update
params:
file: src/serverless-expansion.yml
- command: shell.exec
params:
shell: "bash"
script: |
${PREPARE_SHELL}

if [ -z "${SERVERLESS_MONGODB_VERSION}" ]; then
echo "expected SERVERLESS_MONGODB_VERSION to be set"
exit 1
fi

# Download the enterprise server download for current platform to $MONGODB_BINARIES.
# This is required for tests that need mongocryptd.
# $MONGODB_BINARIES is added to the $PATH in fetch-source.
${PYTHON3_BINARY} $DRIVERS_TOOLS/.evergreen/mongodl.py \
--component archive \
--version ${SERVERLESS_MONGODB_VERSION} \
--edition enterprise \
--out $MONGODB_BINARIES \
--strip-path-components 2

# Download the crypt_shared dynamic library for the current platform.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, this is handled by run-orchestration.sh in drivers-evergreen-tools for non-serverless tests. That appears to be exactly what DRIVERS-2459 is meant to address.

IMO, this should have been addressed as a prerequisite for DRIVERS-2400 before spawning language tickets and inviting drivers to sort this out on their own.

${PYTHON3_BINARY} $DRIVERS_TOOLS/.evergreen/mongodl.py \
--component crypt_shared \
--version ${SERVERLESS_MONGODB_VERSION} \
--edition enterprise \
--out . \
--only "**/mongo_crypt_v1.*" \
--strip-path-components 1

# Find the crypt_shared library file in the current directory and set the CRYPT_SHARED_LIB_PATH to
# the path of that file. Only look for .so, .dll, or .dylib files to prevent matching any other
# downloaded files.
CRYPT_SHARED_LIB_PATH="$(find $(pwd) -maxdepth 1 -type f \
-name 'mongo_crypt_v1.so' -o \
-name 'mongo_crypt_v1.dll' -o \
-name 'mongo_crypt_v1.dylib')"

# If we're on Windows, convert the "cygdrive" path to Windows-style paths.
if [ "Windows_NT" = "$OS" ]; then
CRYPT_SHARED_LIB_PATH=$(cygpath -m $CRYPT_SHARED_LIB_PATH)
fi

echo "CRYPT_SHARED_LIB_PATH: $CRYPT_SHARED_LIB_PATH" >> crypt-expansion.yml

# Load the expansion file to make an evergreen variable with the current unique version
- command: expansions.update
params:
file: crypt-expansion.yml

"delete serverless instance":
- command: shell.exec
Expand Down Expand Up @@ -277,6 +326,19 @@ functions:
working_dir: "src"
script: |
${PREPARE_SHELL}
export AWS_ACCESS_KEY_ID="${client_side_encryption_aws_access_key_id}"
export AWS_SECRET_ACCESS_KEY="${client_side_encryption_aws_secret_access_key}"
export AZURE_TENANT_ID="${client_side_encryption_azure_tenant_id}"
export AZURE_CLIENT_ID="${client_side_encryption_azure_client_id}"
export AZURE_CLIENT_SECRET="${client_side_encryption_azure_client_secret}"
export GCP_EMAIL="${client_side_encryption_gcp_email}"
export GCP_PRIVATE_KEY="${client_side_encryption_gcp_privatekey}"
export KMIP_ENDPOINT="${client_side_encryption_kmip_endpoint}"
export KMS_ENDPOINT_EXPIRED="${client_side_encryption_kms_endpoint_expired}"
export KMS_ENDPOINT_WRONG_HOST="${client_side_encryption_kms_endpoint_wrong_host}"
export KMS_ENDPOINT_REQUIRE_CLIENT_CERT="${client_side_encryption_kms_endpoint_require_client_cert}"
export KMS_TLS_CA_FILE="${client_side_encryption_kms_tls_ca_file}"
export KMS_TLS_CERTIFICATE_KEY_FILE="${client_side_encryption_kms_tls_certificate_key_file}"
export MONGODB_IS_SERVERLESS=on
export MONGODB_USERNAME=${SERVERLESS_ATLAS_USER}
export MONGODB_PASSWORD=${SERVERLESS_ATLAS_PASSWORD}
Expand Down Expand Up @@ -505,6 +567,7 @@ tasks:
tags: ["serverless"]
commands:
- func: "create serverless instance"
- func: "start kms servers"
- func: "run serverless tests"

- name: "test-loadBalanced"
Expand Down
1 change: 1 addition & 0 deletions tests/SpecTests/ClientSideEncryptionSpecTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,7 @@
*
* @see https://github.com/mongodb/specifications/tree/master/source/client-side-encryption
* @group csfle
* @group serverless
*/
class ClientSideEncryptionSpecTest extends FunctionalTestCase
{
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-BypassQueryAnalysis.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

None of the commits in this PR indicate the sync target for spec tests. According to DRIVERS-2400 this is mongodb/specifications@59a07b7. Please add that and ensure it ends up in the final squash message.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yes. Added to the last commit and will make sure it ends in the squash message.

]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-Compact.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-CreateCollection.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-DecryptExistingData.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-Delete.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-EncryptedFields-vs-EncryptedFieldsMap.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-EncryptedFields-vs-jsonSchema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-EncryptedFieldsMap-defaults.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-FindOneAndUpdate.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-InsertFind-Indexed.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-InsertFind-Unindexed.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-MissingKey.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-NoEncryption.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-Update.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down
3 changes: 2 additions & 1 deletion tests/SpecTests/client-side-encryption/tests/fle2-validatorAndPartialFieldExpression.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"minServerVersion": "6.0.0",
"topology": [
"replicaset",
"sharded"
"sharded",
"load-balanced"
]
}
],
Expand Down