Skip to content

CSHARP-3740: Add native support for AWS IAM Roles for service accounts, EKS in particular. #947

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Nov 22, 2022
Merged

CSHARP-3740: Add native support for AWS IAM Roles for service accounts, EKS in particular. #947

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
8d11d50
CSHARP-3740: Add native support for AWS IAM Roles for service account…
DmitryLukyanov Nov 15, 2022
d5de37d
Code review.
DmitryLukyanov Nov 15, 2022
a39940b
Add lock.
DmitryLukyanov Nov 21, 2022
ce1a488
Fix comments
DmitryLukyanov Nov 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 80 additions & 1 deletion evergreen/evergreen.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -432,7 +432,12 @@ functions:
"iam_auth_assume_role_name" : "${iam_auth_assume_role_name}",
"iam_auth_ec2_instance_account" : "${iam_auth_ec2_instance_account}",
"iam_auth_ec2_instance_secret_access_key" : "${iam_auth_ec2_instance_secret_access_key}",
"iam_auth_ec2_instance_profile" : "${iam_auth_ec2_instance_profile}"
"iam_auth_ec2_instance_profile" : "${iam_auth_ec2_instance_profile}",
"iam_auth_assume_web_role_name": "${iam_auth_assume_web_role_name}",
"iam_web_identity_issuer": "${iam_web_identity_issuer}",
"iam_web_identity_rsa_key": "${iam_web_identity_rsa_key}",
"iam_web_identity_jwks_uri": "${iam_web_identity_jwks_uri}",
"iam_web_identity_token_file": "${iam_web_identity_token_file}"
}
EOF

Expand Down Expand Up @@ -551,6 +556,76 @@ functions:
mongo --nodb setup.js aws_e2e_ecs.js
cd -

run-aws-auth-test-with-aws-web-identity-credentials:
- command: shell.exec
type: test
params:
working_dir: mongo-csharp-driver
script: |
${PREPARE_SHELL}
if [ "${skip_web_identity_auth_test}" = "true" ]; then
echo "This platform does not support the web identity auth test, skipping..."
exit 0
fi
cd ${DRIVERS_TOOLS}/.evergreen/auth_aws
. ./activate_venv.sh
mongo aws_e2e_web_identity.js
- command: shell.exec
type: test
params:
working_dir: mongo-csharp-driver
silent: true
script: |
if [ "${skip_web_identity_auth_test}" = "true" ]; then
echo "This platform does not support the web identity auth test, skipping..."
exit 0
fi
# DO NOT ECHO WITH XTRACE (which PREPARE_SHELL does)
cat <<'EOF' > "${PROJECT_DIRECTORY}/prepare_mongodb_aws.sh"
export AWS_ROLE_ARN="${iam_auth_assume_web_role_name}"
export AWS_WEB_IDENTITY_TOKEN_FILE="${iam_web_identity_token_file}"
export MONGODB_URI="mongodb://localhost"
EOF
- command: shell.exec
type: test
params:
working_dir: mongo-csharp-driver
script: |
${PREPARE_SHELL}
if [ "${skip_web_identity_auth_test}" = "true" ]; then
echo "This platform does not support the web identity auth test, skipping..."
exit 0
fi
PROJECT_DIRECTORY=${PROJECT_DIRECTORY} OS=$OS ASSERT_NO_URI_CREDS=true evergreen/run-mongodb-aws-test.sh
- command: shell.exec
type: test
params:
working_dir: mongo-csharp-driver
silent: true
script: |
if [ "${skip_EC2_auth_test}" = "true" ]; then
echo "This platform does not support the web identity auth test, skipping..."
exit 0
fi
# DO NOT ECHO WITH XTRACE (which PREPARE_SHELL does)
cat <<'EOF' > "${PROJECT_DIRECTORY}/prepare_mongodb_aws.sh"
export AWS_ROLE_ARN="${iam_auth_assume_web_role_name}"
export AWS_WEB_IDENTITY_TOKEN_FILE="${iam_web_identity_token_file}"
export AWS_ROLE_SESSION_NAME="test"
export MONGODB_URI="mongodb://localhost"
EOF
- command: shell.exec
type: test
params:
working_dir: mongo-csharp-driver
script: |
${PREPARE_SHELL}
if [ "${skip_web_identity_auth_test}" = "true" ]; then
echo "This platform does not support the web identity auth test, skipping..."
exit 0
fi
PROJECT_DIRECTORY=${PROJECT_DIRECTORY} OS=$OS ASSERT_NO_URI_CREDS=true evergreen/run-mongodb-aws-test.sh

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we running run-mongodb-aws-test.sh twice, once without AWS_ROLE_SESSION_NAME set and once with it set? If we are testing two different scenarios, we should probably separate this into two separate functions. Otherwise if one fails, it will not be immediately obvious which one. We would have to check whether AWS_ROLE_SESSION_NAME is set or not to understand which test case failed. Thoughts?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these cases are very similar (but different) because if AWS_ROLE_SESSION_NAME is missed, it will be added automatically by sdk. Also the reason why we configure both cases is requirement in the spec here (supporting both cases where AWS_ROLE_SESSION_NAME is added and where not)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

into two separate functions.

both of these steps require the same precondition in the first step, so I would say it's not too wrong running these ones in the same scope.

it will not be immediately obvious which one

given that it's almost the same scenarios, I would not expect failing them differently one from another. But even if so, EG provides a way to know what particular step was failed. For example pay attention on (step 9.2.. here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair enough.

run-aws-auth-test-with-aws-credentials-as-environment-variables:
- command: shell.exec
type: test
Expand Down Expand Up @@ -1067,6 +1142,7 @@ tasks:
- func: run-aws-auth-test-with-aws-credentials-and-session-token-as-environment-variables
- func: run-aws-auth-test-with-aws-EC2-credentials
- func: run-aws-auth-test-with-aws-ECS-credentials
- func: run-aws-auth-test-with-aws-web-identity-credentials

- name: stable-api-tests-net472
commands:
Expand Down Expand Up @@ -1599,6 +1675,7 @@ axes:
variables:
OS: "windows-64"
skip_ECS_auth_test: true
skip_web_identity_auth_test: true
run_on: windows-64-vs2017-test
- id: "ubuntu-1804"
display_name: "Ubuntu 18.04"
Expand All @@ -1611,13 +1688,15 @@ axes:
OS: "macos-1100"
skip_EC2_auth_test: true
skip_ECS_auth_test: true
skip_web_identity_auth_test: true
run_on: macos-1100
- id: "macos-1100-arm64"
display_name: "macOS 11.00 M1"
variables:
OS: "macos-1100-arm64"
skip_EC2_auth_test: true
skip_ECS_auth_test: true
skip_web_identity_auth_test: true
run_on: macos-1100-arm64

- id: topology
Expand Down
6 changes: 3 additions & 3 deletions evergreen/run-mongodb-aws-ecs-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@ set -o errexit # Exit the script with error if any of the commands fail
# ASSERT_NO_URI_CREDS Determines whether we need assert existence credentials in connection string or not
#
# Environment variables used as output:
# AWS_TESTS_ENABLED Allows runnings AWS tests
# AWS_ECS_TEST Allows runnings EVS tests
# AWS_TESTS_ENABLED Allows running AWS tests
# AWS_ECS_ENABLED Allows running ECS tests
#
############################################
# Main Program #
Expand All @@ -28,7 +28,7 @@ fi
# Now we can safely enable xtrace
set -o xtrace
export AWS_TESTS_ENABLED=true
export AWS_ECS_TEST=true
export AWS_ECS_ENABLED=true

# EG scripts for ECS assume that a root folder is "src" and all driver side scripts are placed in ".evergreen" folder.
# So that script is copied into "src/.evergreen" before running
Expand Down
Loading