Skip to content

Add unified, legacy, and prose tests for CSFLE multi-KMS support. #1424

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Jul 25, 2024
Merged

Add unified, legacy, and prose tests for CSFLE multi-KMS support. #1424

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
8f5686c
Add unified, legacy, and prose tests for CSFLE multi-KMS support.
vbabanin Jun 20, 2024
c75aaf3
Rename variable to convey the purpose of the usage.
vbabanin Jun 22, 2024
b363a27
Add JavaDoc and additional tests.
vbabanin Jun 22, 2024
aa433fc
Apply suggestions from code review
vbabanin Jul 17, 2024
39d9ac6
Add javadoc.
vbabanin Jul 17, 2024
2395f26
Make opts required.
vbabanin Jul 17, 2024
967c046
Remove redundant method and variable.
vbabanin Jul 20, 2024
7318290
Merge branch 'master' into JAVA-5275
vbabanin Jul 24, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .evergreen/.evg.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,8 @@ functions:
env:
AWS_ACCESS_KEY_ID: ${aws_access_key_id}
AWS_SECRET_ACCESS_KEY: ${aws_secret_access_key}
AWS_ACCESS_KEY_ID_AWS_KMS_NAMED: ${aws_access_key_id_2}
AWS_SECRET_ACCESS_KEY_AWS_KMS_NAMED: ${aws_secret_access_key_2}
AWS_DEFAULT_REGION: us-east-1
AZURE_TENANT_ID: ${azure_tenant_id}
AZURE_CLIENT_ID: ${azure_client_id}
Expand Down Expand Up @@ -708,6 +710,8 @@ functions:
env:
AWS_ACCESS_KEY_ID: ${aws_access_key_id}
AWS_SECRET_ACCESS_KEY: ${aws_secret_access_key}
AWS_ACCESS_KEY_ID_AWS_KMS_NAMED: ${aws_access_key_id_2}
AWS_SECRET_ACCESS_KEY_AWS_KMS_NAMED: ${aws_secret_access_key_2}
AWS_DEFAULT_REGION: us-east-1
AZURE_TENANT_ID: ${azure_tenant_id}
AZURE_CLIENT_ID: ${azure_client_id}
Expand Down
30 changes: 16 additions & 14 deletions .evergreen/run-csfle-tests-with-mongocryptd.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,20 +4,22 @@ set -o xtrace # Write all commands first to stderr
set -o errexit # Exit the script with error if any of the commands fail

# Supported/used environment variables:
# MONGODB_URI Set the suggested connection MONGODB_URI (including credentials and topology info)
# JAVA_VERSION Set the version of java to be used. Java versions can be set from the java toolchain /opt/java
# AWS_ACCESS_KEY_ID The AWS access key identifier for client-side encryption
# AWS_SECRET_ACCESS_KEY The AWS secret access key for client-side encryption
# AWS_TEMP_ACCESS_KEY_ID The temporary AWS access key identifier for client-side encryption
# AWS_TEMP_SECRET_ACCESS_KEY The temporary AWS secret access key for client-side encryption
# AWS_TEMP_SESSION_TOKEN The temporary AWS session token for client-side encryption
# AZURE_TENANT_ID The Azure tenant identifier for client-side encryption
# AZURE_CLIENT_ID The Azure client identifier for client-side encryption
# AZURE_CLIENT_SECRET The Azure client secret for client-side encryption
# GCP_EMAIL The GCP email for client-side encryption
# GCP_PRIVATE_KEY The GCP private key for client-side encryption
# AZUREKMS_KEY_VAULT_ENDPOINT The Azure key vault endpoint for integration tests
# AZUREKMS_KEY_NAME The Azure key name endpoint for integration tests
# MONGODB_URI Set the suggested connection MONGODB_URI (including credentials and topology info)
# JAVA_VERSION Set the version of java to be used. Java versions can be set from the java toolchain /opt/java
# AWS_ACCESS_KEY_ID The AWS access key identifier for client-side encryption
# AWS_SECRET_ACCESS_KEY The AWS secret access key for client-side encryption
# AWS_ACCESS_KEY_ID_AWS_KMS_NAMED The AWS access key identifier for client-side encryption's named KMS provider.
# AWS_SECRET_ACCESS_KEY_AWS_KMS_NAMED The AWS secret access key for client-side encryption's named KMS provider.
# AWS_TEMP_ACCESS_KEY_ID The temporary AWS access key identifier for client-side encryption
# AWS_TEMP_SECRET_ACCESS_KEY The temporary AWS secret access key for client-side encryption
# AWS_TEMP_SESSION_TOKEN The temporary AWS session token for client-side encryption
# AZURE_TENANT_ID The Azure tenant identifier for client-side encryption
# AZURE_CLIENT_ID The Azure client identifier for client-side encryption
# AZURE_CLIENT_SECRET The Azure client secret for client-side encryption
# GCP_EMAIL The GCP email for client-side encryption
# GCP_PRIVATE_KEY The GCP private key for client-side encryption
# AZUREKMS_KEY_VAULT_ENDPOINT The Azure key vault endpoint for integration tests
# AZUREKMS_KEY_NAME The Azure key name endpoint for integration tests

MONGODB_URI=${MONGODB_URI:-}

Expand Down
46 changes: 24 additions & 22 deletions .evergreen/run-tests.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,28 +4,30 @@ set -o xtrace # Write all commands first to stderr
set -o errexit # Exit the script with error if any of the commands fail

# Supported/used environment variables:
# AUTH Set to enable authentication. Values are: "auth" / "noauth" (default)
# SSL Set to enable SSL. Values are "ssl" / "nossl" (default)
# NETTY_SSL_PROVIDER The Netty TLS/SSL protocol provider. Ignored unless SSL is "ssl" and STREAM_TYPE is "netty". Values are "JDK", "OPENSSL", null (a.k.a. "" or '') (default).
# MONGODB_URI Set the suggested connection MONGODB_URI (including credentials and topology info)
# TOPOLOGY Allows you to modify variables and the MONGODB_URI based on test topology
# Supported values: "server", "replica_set", "sharded_cluster"
# COMPRESSOR Set to enable compression. Values are "snappy" and "zlib" (default is no compression)
# STREAM_TYPE Set the stream type. Values are "nio2" or "netty". Defaults to "nio2".
# JDK Set the version of java to be used. Java versions can be set from the java toolchain /opt/java
# SLOW_TESTS_ONLY Set to true to only run the slow tests
# AWS_ACCESS_KEY_ID The AWS access key identifier for client-side encryption
# AWS_SECRET_ACCESS_KEY The AWS secret access key for client-side encryption
# AWS_TEMP_ACCESS_KEY_ID The temporary AWS access key identifier for client-side encryption
# AWS_TEMP_SECRET_ACCESS_KEY The temporary AWS secret access key for client-side encryption
# AWS_TEMP_SESSION_TOKEN The temporary AWS session token for client-side encryption
# AZURE_TENANT_ID The Azure tenant identifier for client-side encryption
# AZURE_CLIENT_ID The Azure client identifier for client-side encryption
# AZURE_CLIENT_SECRET The Azure client secret for client-side encryption
# GCP_EMAIL The GCP email for client-side encryption
# GCP_PRIVATE_KEY The GCP private key for client-side encryption
# AZUREKMS_KEY_VAULT_ENDPOINT The Azure key vault endpoint for integration tests
# AZUREKMS_KEY_NAME The Azure key name endpoint for integration tests
# AUTH Set to enable authentication. Values are: "auth" / "noauth" (default)
# SSL Set to enable SSL. Values are "ssl" / "nossl" (default)
# NETTY_SSL_PROVIDER The Netty TLS/SSL protocol provider. Ignored unless SSL is "ssl" and STREAM_TYPE is "netty". Values are "JDK", "OPENSSL", null (a.k.a. "" or '') (default).
# MONGODB_URI Set the suggested connection MONGODB_URI (including credentials and topology info)
# TOPOLOGY Allows you to modify variables and the MONGODB_URI based on test topology
# Supported values: "server", "replica_set", "sharded_cluster"
# COMPRESSOR Set to enable compression. Values are "snappy" and "zlib" (default is no compression)
# STREAM_TYPE Set the stream type. Values are "nio2" or "netty". Defaults to "nio2".
# JDK Set the version of java to be used. Java versions can be set from the java toolchain /opt/java
# SLOW_TESTS_ONLY Set to true to only run the slow tests
# AWS_ACCESS_KEY_ID The AWS access key identifier for client-side encryption
# AWS_SECRET_ACCESS_KEY The AWS secret access key for client-side encryption
# AWS_ACCESS_KEY_ID_AWS_KMS_NAMED The AWS access key identifier for client-side encryption's named KMS provider.
# AWS_SECRET_ACCESS_KEY_AWS_KMS_NAMED The AWS secret access key for client-side encryption's named KMS provider.
# AWS_TEMP_ACCESS_KEY_ID The temporary AWS access key identifier for client-side encryption
# AWS_TEMP_SECRET_ACCESS_KEY The temporary AWS secret access key for client-side encryption
# AWS_TEMP_SESSION_TOKEN The temporary AWS session token for client-side encryption
# AZURE_TENANT_ID The Azure tenant identifier for client-side encryption
# AZURE_CLIENT_ID The Azure client identifier for client-side encryption
# AZURE_CLIENT_SECRET The Azure client secret for client-side encryption
# GCP_EMAIL The GCP email for client-side encryption
# GCP_PRIVATE_KEY The GCP private key for client-side encryption
# AZUREKMS_KEY_VAULT_ENDPOINT The Azure key vault endpoint for integration tests
# AZUREKMS_KEY_NAME The Azure key name endpoint for integration tests

AUTH=${AUTH:-noauth}
SSL=${SSL:-nossl}
Expand Down
13 changes: 9 additions & 4 deletions driver-core/src/main/com/mongodb/AutoEncryptionSettings.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -293,9 +293,14 @@ public String getKeyVaultNamespace() {
/**
* Gets the map of KMS provider properties.
*
* <p> Multiple KMS providers can be specified within this map. Each KMS provider is identified by a unique key.
* Keys are formatted as either {@code "KMS provider type"} or {@code "KMS provider type:KMS provider name"} (e.g., "aws" or "aws:myname").
* <p>
* Multiple KMS providers may be specified. The following KMS providers are supported: "aws", "azure", "gcp" and "local". The
* kmsProviders map values differ by provider:
* Supported KMS provider types include "aws", "azure", "gcp", and "local". The provider name is optional and allows
* for the configuration of multiple providers of the same type under different names (e.g., "aws:name1" and
* "aws:name2" could represent different AWS accounts).
* <p>
* The kmsProviders map values differ by provider type. The following properties are supported for each provider type:
* </p>
* <p>
* For "aws", the properties are:
Expand Down Expand Up @@ -335,15 +340,15 @@ public String getKeyVaultNamespace() {
* <ul>
* <li>key: byte[] of length 96, the local key</li>
* </ul>
*
* <p>
* It is also permitted for the value of a kms provider to be an empty map, in which case the driver will first
* </p>
* <ul>
* <li>use the {@link Supplier} configured in {@link #getKmsProviderPropertySuppliers()} to obtain a non-empty map</li>
* <li>attempt to obtain the properties from the environment</li>
* </ul>
*
* However, KMS providers containing a name (e.g., "aws:myname") does not support dynamically obtaining KMS properties from the {@link Supplier}
* or environment.
* @return map of KMS provider properties
* @see #getKmsProviderPropertySuppliers()
*/
Expand Down
11 changes: 9 additions & 2 deletions driver-core/src/main/com/mongodb/ClientEncryptionSettings.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -169,9 +169,14 @@ public String getKeyVaultNamespace() {
/**
* Gets the map of KMS provider properties.
*
* <p> Multiple KMS providers can be specified within this map. Each KMS provider is identified by a unique key.
* Keys are formatted as either {@code "KMS provider type"} or {@code "KMS provider type:KMS provider name"} (e.g., "aws" or "aws:myname").
* <p>
* Multiple KMS providers may be specified. The following KMS providers are supported: "aws", "azure", "gcp" and "local". The
* kmsProviders map values differ by provider:
* Supported KMS provider types include "aws", "azure", "gcp", and "local". The provider name is optional and allows
* for the configuration of multiple providers of the same type under different names (e.g., "aws:name1" and
* "aws:name2" could represent different AWS accounts).
* <p>
* The kmsProviders map values differ by provider type. The following properties are supported for each provider type:
* </p>
* <p>
* For "aws", the properties are:
Expand Down Expand Up @@ -218,6 +223,8 @@ public String getKeyVaultNamespace() {
* <li>use the {@link Supplier} configured in {@link #getKmsProviderPropertySuppliers()} to obtain a non-empty map</li>
* <li>attempt to obtain the properties from the environment</li>
* </ul>
* However, KMS providers containing a name (e.g., "aws:myname") does not support dynamically obtaining KMS properties from the {@link Supplier}
* or environment.
* @return map of KMS provider properties
* @see #getKmsProviderPropertySuppliers()
*/
Expand Down
10 changes: 5 additions & 5 deletions driver-core/src/main/com/mongodb/client/model/vault/DataKeyOptions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,22 +95,22 @@ public List<String> getKeyAltNames() {
* The masterKey identifies a KMS-specific key used to encrypt the new data key.
* </p>
* <p>
* If the kmsProvider is "aws" the master key is required and must contain the following fields:
* If the kmsProvider type is "aws" the master key is required and must contain the following fields:
* </p>
* <ul>
* <li>region: a String containing the AWS region in which to locate the master key</li>
* <li>key: a String containing the Amazon Resource Name (ARN) to the AWS customer master key</li>
* </ul>
* <p>
* If the kmsProvider is "azure" the master key is required and must contain the following fields:
* If the kmsProvider type is "azure" the master key is required and must contain the following fields:
* </p>
* <ul>
* <li>keyVaultEndpoint: a String with the host name and an optional port. Example: "example.vault.azure.net".</li>
* <li>keyName: a String</li>
* <li>keyVersion: an optional String, the specific version of the named key, defaults to using the key's primary version.</li>
* </ul>
* <p>
* If the kmsProvider is "gcp" the master key is required and must contain the following fields:
* If the kmsProvidertype type is "gcp" the master key is required and must contain the following fields:
* </p>
* <ul>
* <li>projectId: a String</li>
Expand All @@ -121,7 +121,7 @@ public List<String> getKeyAltNames() {
* <li>endpoint: an optional String, with the host with optional port. Defaults to "cloudkms.googleapis.com".</li>
* </ul>
* <p>
* If the kmsProvider is "kmip" the master key is required and must contain the following fields:
* If the kmsProvider type is "kmip" the master key is required and must contain the following fields:
* </p>
* <ul>
* <li>keyId: optional String, keyId is the KMIP Unique Identifier to a 96 byte KMIP Secret Data managed object. If keyId is
Expand All @@ -133,7 +133,7 @@ public List<String> getKeyAltNames() {
* to false.</li>
* </ul>
* <p>
* If the kmsProvider is "local" the masterKey is not applicable.
* If the kmsProvider type is "local" the masterKey is not applicable.
* </p>
* @return the master key document
*/
Expand Down
10 changes: 5 additions & 5 deletions driver-core/src/main/com/mongodb/client/model/vault/RewrapManyDataKeyOptions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,22 +73,22 @@ public RewrapManyDataKeyOptions masterKey(final BsonDocument masterKey) {
* The masterKey identifies a KMS-specific key used to encrypt the new data key.
* </p>
* <p>
* If the kmsProvider is "aws" the master key is required and must contain the following fields:
* If the kmsProvider type is "aws" the master key is required and must contain the following fields:
* </p>
* <ul>
* <li>region: a String containing the AWS region in which to locate the master key</li>
* <li>key: a String containing the Amazon Resource Name (ARN) to the AWS customer master key</li>
* </ul>
* <p>
* If the kmsProvider is "azure" the master key is required and must contain the following fields:
* If the kmsProvider type is "azure" the master key is required and must contain the following fields:
* </p>
* <ul>
* <li>keyVaultEndpoint: a String with the host name and an optional port. Example: "example.vault.azure.net".</li>
* <li>keyName: a String</li>
* <li>keyVersion: an optional String, the specific version of the named key, defaults to using the key's primary version.</li>
* </ul>
* <p>
* If the kmsProvider is "gcp" the master key is required and must contain the following fields:
* If the kmsProvider type is "gcp" the master key is required and must contain the following fields:
* </p>
* <ul>
* <li>projectId: a String</li>
Expand All @@ -99,7 +99,7 @@ public RewrapManyDataKeyOptions masterKey(final BsonDocument masterKey) {
* <li>endpoint: an optional String, with the host with optional port. Defaults to "cloudkms.googleapis.com".</li>
* </ul>
* <p>
* If the kmsProvider is "kmip" the master key is required and must contain the following fields:
* If the kmsProvider type is "kmip" the master key is required and must contain the following fields:
* </p>
* <ul>
* <li>keyId: optional String, keyId is the KMIP Unique Identifier to a 96 byte KMIP Secret Data managed object. If keyId is
Expand All @@ -108,7 +108,7 @@ public RewrapManyDataKeyOptions masterKey(final BsonDocument masterKey) {
* defaults to the required endpoint from the KMS providers map.</li>
* </ul>
* <p>
* If the kmsProvider is "local" the masterKey is not applicable.
* If the kmsProvider is type "local" the masterKey is not applicable.
* </p>
* @return the master key document
*/
Expand Down
Loading