Skip to content

PHPC-2356: Remove explicit env variables #1514

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Feb 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file removed .evergreen/atlas-uris.txt.enc
Binary file not shown.
42 changes: 25 additions & 17 deletions .evergreen/config/functions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -167,9 +167,19 @@ functions:
"bootstrap mongo-orchestration":
- command: shell.exec
params:
include_expansions_in_env:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noted that vars (when calling a func) actually define expansions (per Functions). That distinction wasn't clear to me before this PR.

- MONGODB_VERSION
- TOPOLOGY
- AUTH
- SSL
- STORAGE_ENGINE
- LOAD_BALANCER
- REQUIRE_API_VERSION
- ORCHESTRATION_FILE
script: |
${PREPARE_SHELL}
SKIP_LEGACY_SHELL=true MONGODB_VERSION=${MONGODB_VERSION} TOPOLOGY=${TOPOLOGY} AUTH=${AUTH} SSL=${SSL} STORAGE_ENGINE=${STORAGE_ENGINE} LOAD_BALANCER=${LOAD_BALANCER} REQUIRE_API_VERSION=${REQUIRE_API_VERSION} ORCHESTRATION_FILE=${ORCHESTRATION_FILE} sh ${DRIVERS_TOOLS}/.evergreen/run-orchestration.sh
SKIP_LEGACY_SHELL=true \
sh ${DRIVERS_TOOLS}/.evergreen/run-orchestration.sh
# run-orchestration generates expansion file with MONGODB_URI and CRYPT_SHARED_LIB_PATH
- command: expansions.update
params:
Expand All @@ -185,28 +195,32 @@ functions:
"run OCSP responder":
- command: shell.exec
params:
include_expansions_in_env:
- TEST_COLUMN
- CERT_TYPE
- USE_DELEGATE
shell: bash
script: |
${PREPARE_SHELL}
TEST_COLUMN=${TEST_COLUMN} \
CERT_TYPE=${CERT_TYPE} \
USE_DELEGATE=${USE_DELEGATE} \
${PROJECT_DIRECTORY}/.evergreen/run-ocsp-responder.sh

"run tests":
- command: shell.exec
type: test
params:
include_expansions_in_env:
- API_VERSION
- ATLAS_CONNECTIVITY_URIS
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this always be specified? In the test-atlas-connectivity task, run tests is invoked with vars: { TESTS: "tests/atlas.phpt" }, so I would expect that's the only case where you'd actually want to pass this potentially sensitive environment variable onwards.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could create a separate function to run just the connectivity tests that also passes the necessary expansion to the env, but this would entail duplicating all of the logic in this function. I don't think it's worth it, as any attacker who might modify tests to get access to this environment variable would be able to do so in the particular test that includes this variable.

- CRYPT_SHARED_LIB_PATH
- MONGODB_URI
- APPEND_URI
- SKIP_CRYPT_SHARED
- SSL
- SSL_DIR
- TESTS
working_dir: "src"
script: |
${PREPARE_SHELL}
API_VERSION=${API_VERSION} \
CRYPT_SHARED_LIB_PATH=${CRYPT_SHARED_LIB_PATH} \
MONGODB_URI="${MONGODB_URI}${APPEND_URI}" \
SKIP_CRYPT_SHARED=${SKIP_CRYPT_SHARED} \
SSL=${SSL} \
SSL_DIR=${SSL_DIR} \
TESTS=${TESTS} \
sh ${PROJECT_DIRECTORY}/.evergreen/run-tests.sh

"cleanup":
Expand Down Expand Up @@ -242,12 +256,6 @@ functions:
# Don't use ${file} syntax here because evergreen treats it as an empty expansion.
[ -f "$file" ] && sh $file || echo "$file not available, skipping"

"decrypt atlas URI list":
- command: shell.exec
params:
script: |
openssl aes-256-cbc -S "${encrypted_uris_salt}" -K "${encrypted_uris_key}" -iv "${encrypted_uris_iv}" -in ${PROJECT_DIRECTORY}/.evergreen/atlas-uris.txt.enc -out ${PROJECT_DIRECTORY}/.evergreen/atlas-uris.txt -d
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes it available for us to use in tests without having to decrypt a file and potentially leak the encryption credentials.

Are other drivers doing this as well? I don't have an opinion on which is more secure, but storing the URIs directly in the Evergreen project config is certainly more convenient.

Note: you can also remove .evergreen/atlas-uris.txt from .gitignore.


"start load balancer":
- command: shell.exec
params:
Expand Down
1 change: 0 additions & 1 deletion .evergreen/config/test-tasks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ tasks:
- name: "test-atlas-connectivity"
tags: ["atlas", "nodb"]
commands:
- func: "decrypt atlas URI list"
- func: "run tests"
vars:
TESTS: "tests/atlas.phpt"
Expand Down
2 changes: 1 addition & 1 deletion .evergreen/config/test-variants.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ buildvariants:
- ".sharded .local !.3.6 !.4.0 !.4.2 !.4.4 !.5.0"
- ".loadbalanced .local !.3.6 !.4.0 !.4.2 !.4.4 !.5.0"
- "test-atlas-connectivity"
- ".ocsp !.4.4"
- ".ocsp !.4.4 !.5.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, this drops OCSP testing on 5.0. What the impetus for this change? The OCSP spec test README doesn't talk about server versions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We only test 6.0 and newer on Debian 11 - 5.0 and older are tested on Debian 9.2. OCSP tests for 5.0 are included in the test-debian92-php82-local matrix below.

display_tasks:
- name: "test-ocsp-latest"
execution_tasks:
Expand Down
4 changes: 4 additions & 0 deletions .evergreen/run-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ set -o errexit # Exit the script with error if any of the commands fail
API_VERSION=${API_VERSION:-} # Optional API_VERSION environment variable for run-tests.php
CRYPT_SHARED_LIB_PATH="${CRYPT_SHARED_LIB_PATH:-}" # Optional path to crypt_shared library
MONGODB_URI=${MONGODB_URI:-} # Connection string (including credentials and topology info)
APPEND_URI=${APPEND_URI:-} # Additional URI options to append to the URI. Used in OCSP tests
SKIP_CRYPT_SHARED="${SKIP_CRYPT_SHARED:-no}" # Specify "yes" to ignore CRYPT_SHARED_LIB_PATH. Defaults to "no"
SSL=${SSL:-no} # Specify "yes" to enable SSL. Defaults to "no"
SSL_DIR=${SSL_DIR-} # Optional SSL_DIR environment variable for run-tests.php
Expand All @@ -15,6 +16,9 @@ OS=$(uname -s | tr '[:upper:]' '[:lower:]')

export REPORT_EXIT_STATUS=1

# Append test-specific URI options
MONGODB_URI=${MONGODB_URI}${APPEND_URI}

# Determine if MONGODB_URI already has a query string
SUFFIX=$(echo "$MONGODB_URI" | grep -Eo "\?(.*)" | cat)

Expand Down
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,6 @@ php.ini
!.github/
!tests/utils/*.php
!.evergreen
.evergreen/atlas-uris.txt


!scripts
Expand Down
18 changes: 5 additions & 13 deletions tests/atlas.phpt
Original file line number Diff line number Diff line change
@@ -1,25 +1,17 @@
--TEST--
Atlas Connectivity Tests
--SKIPIF--
<?php
if (!file_exists('.evergreen/atlas-uris.txt')) { echo "skip Atlas URIs not found\n"; }
if (filesize('.evergreen/atlas-uris.txt') < 10) { echo "skip Atlas URI file empty\n"; }
if (getenv('TESTS') !== 'tests/atlas.phpt') { echo "skip Atlas tests not wanted\n"; }
?>
<?php require __DIR__ . "/utils/basic-skipif.inc"; ?>
<?php skip_if_no_atlas_connectivity_urls(); ?>
<?php if (getenv('TESTS') !== 'tests/atlas.phpt') { die('skip Atlas tests not wanted'); } ?>
--FILE--
<?php
$urls = explode("\n", file_get_contents('.evergreen/atlas-uris.txt'));
require_once __DIR__ . "/utils/basic.inc";

$command = new \MongoDB\Driver\Command(['ping' => 1]);
$query = new \MongoDB\Driver\Query([]);

foreach ($urls as $url) {
$url = trim($url);

if ($url == '') {
continue;
}

foreach (getAtlasConnectivityUrls() as $url) {
if (strpos($url, '#') === 0) {
echo trim(substr($url, 1)), "\n";
continue;
Expand Down
7 changes: 7 additions & 0 deletions tests/utils/skipif.php
Original file line number Diff line number Diff line change
Expand Up @@ -494,3 +494,10 @@ function skip_if_no_crypt_shared()
exit('skip crypt_shared is not available');
}
}

function skip_if_no_atlas_connectivity_urls()
{
if (getAtlasConnectivityUrls() === []) {
exit('skip No Atlas URIs found');
}
}
23 changes: 23 additions & 0 deletions tests/utils/tools.php
Original file line number Diff line number Diff line change
Expand Up @@ -853,3 +853,26 @@ function failGetMore(Manager $manager)

throw new Exception("Trying to configure a getMore fail point for a server version ($version) that doesn't support it");
}

function getAtlasConnectivityUrls(): array
{
$atlasUriString = getenv('ATLAS_CONNECTIVITY_URIS') ?: '';
if (!$atlasUriString) {
return [];
}

$rawUrls = explode("\n", $atlasUriString);

$urls = [];
foreach ($rawUrls as $url) {
$url = trim($url);

if ($url == '') {
continue;
}

$urls[] = $url;
}

return $urls;
}