Skip to content

Add release instructions for static analysis and 3rd party deps (CXX-3009, CXX-3023) #1143

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Jun 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions .evergreen/check-augmented-sbom.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
#!/usr/bin/env bash

set -o errexit
set -o pipefail

command -v podman >/dev/null || {
echo "missing required program podman" 1>&2
exit 1
}

command -v jq >/dev/null || {
echo "missing required program jq" 1>&2
exit 1
}

podman login --password-stdin --username "${ARTIFACTORY_USER:?}" artifactory.corp.mongodb.com <<<"${ARTIFACTORY_PASSWORD:?}"

silkbomb_download_flags=(
# Avoid bumping version or timestamp in diff.
--no-update-sbom-version
--no-update-timestamp

--silk-asset-group mongo-cxx-driver
-o /pwd/etc/augmented.sbom.json.new
)

podman run \
--env-file <(
echo "SILK_CLIENT_ID=${SILK_CLIENT_ID:?}"
echo "SILK_CLIENT_SECRET=${SILK_CLIENT_SECRET:?}"
) \
-it --rm -v "$(pwd):/pwd" \
artifactory.corp.mongodb.com/release-tools-container-registry-public-local/silkbomb:1.0 \
download "${silkbomb_download_flags[@]:?}"

[[ -f ./etc/augmented.sbom.json.new ]] || {
echo "failed to download Augmented SBOM from Silk" 1>&2
exit 1
}

echo "Comparing Augmented SBOM..."

# Allow task to upload the augmented SBOM despite failed diff.
if ! diff -sty --left-column -W 200 \
<(jq ./etc/augmented.sbom.json) \
<(jq ./etc/augmented.sbom.json.new) >|diff.txt; then
declare status
status='{"status":"failed", "type":"test", "should_continue":true, "desc":"detected significant changes in Augmented SBOM"}'
curl -sS -d "${status:?}" -H "Content-Type: application/json" -X POST localhost:2285/task_status || true
fi

cat diff.txt

echo "Comparing Augmented SBOM... done."
50 changes: 50 additions & 0 deletions .mci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -700,6 +700,43 @@ functions:
content_type: text/html
display_name: Scan Build Report

"check augmented sbom":
- command: subprocess.exec
type: test
params:
working_dir: "mongo-cxx-driver"
binary: bash
include_expansions_in_env:
- ARTIFACTORY_USER
- ARTIFACTORY_PASSWORD
- SILK_CLIENT_ID
- SILK_CLIENT_SECRET
args:
- -c
- .evergreen/check-augmented-sbom.sh

"upload augmented sbom":
- command: s3.put
params:
aws_key: ${aws_key}
aws_secret: ${aws_secret}
remote_file: ${project}/${build_variant}/${revision}/${version_id}/${build_id}/silk/augmented.sbom.json
bucket: mciuploads
permissions: public-read
local_file: mongo-cxx-driver/etc/augmented.sbom.json.new
content_type: application/json
display_name: Augmented SBOM
- command: s3.put
params:
aws_key: ${aws_key}
aws_secret: ${aws_secret}
remote_file: ${project}/${build_variant}/${revision}/${version_id}/${build_id}/silk/augmented.sbom.json.diff
bucket: mciuploads
permissions: public-read
local_file: mongo-cxx-driver/diff.txt
content_type: application/json
display_name: Augmented SBOM (Diff)

#######################################
# Post Task #
#######################################
Expand Down Expand Up @@ -1230,6 +1267,14 @@ tasks:
BSONCXX_POLYFILL: std
- func: "upload scan artifacts"

- name: silk-check-augmented-sbom
run_on: rhel8-latest-small
tags: [silk]
commands:
- func: "setup"
- func: "check augmented sbom"
- func: "upload augmented sbom"

task_groups:
- name: tg-abi-stability
max_hosts: -1
Expand Down Expand Up @@ -2171,3 +2216,8 @@ buildvariants:
display_name: scan-build-matrix
tasks:
- name: .scan-build-matrix

- name: silk
display_name: silk
tasks:
- name: .silk
113 changes: 113 additions & 0 deletions etc/augmented.sbom.json
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we get confirmation that this file should be committed rather than generated on-the-fly for releases? The augmented SBOM in Silk will be updated asynchronously and automatically as vulns are discovered by scanning tools, so a committed version would easily become out-of-date repeatedly.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It only needs to be available as an artifact for a given release, and it doesn't need to be regularly committed either. Therefore, the proposed release instructions state that this file only needs to be updated if there are significant changes in the latest Augmented SBOM relative to what is currently committed (as will be tracked by the EVG task). In effect, its presence in the repo isn't just for reporting purposes, but also to enable the tracking of updates to the Augmented SBOM via the EVG task.

Original file line number Diff line number Diff line change
@@ -0,0 +1,113 @@
{
"components": [
{
"bom-ref": "pkg:github/mnmlstc/[email protected]",
"copyright": "Copyright \u00a9 2013 - 2014 MNMLSTC",
"externalReferences": [
{
"type": "distribution",
"url": "https://github.com/mnmlstc/core/archive/refs/tags/v1.1.0.tar.gz"
},
{
"type": "website",
"url": "https://github.com/mnmlstc/core/tree/v1.1.0"
}
],
"group": "mnmlstc",
"licenses": [
{
"license": {
"id": "Apache-2.0"
}
}
],
"name": "core",
"purl": "pkg:github/mnmlstc/[email protected]",
"type": "library",
"version": "v1.1.0"
},
{
"bom-ref": "pkg:github/mongodb/[email protected]",
"copyright": "Copyright 2009-present MongoDB, Inc.",
"externalReferences": [
{
"type": "distribution",
"url": "https://github.com/mongodb/mongo-c-driver/archive/refs/tags/v1.25.0.tar.gz"
},
{
"type": "website",
"url": "https://github.com/mongodb/mongo-c-driver/tree/v1.25.0"
}
],
"group": "mongodb",
"licenses": [
{
"license": {
"id": "Apache-2.0"
}
}
],
"name": "mongo-c-driver",
"purl": "pkg:github/mongodb/[email protected]",
"type": "library",
"version": "v1.25.0"
}
],
"dependencies": [
{
"ref": "pkg:github/mnmlstc/[email protected]"
},
{
"ref": "pkg:github/mongodb/[email protected]"
}
],
"metadata": {
"timestamp": "2024-06-06T18:10:31.789025+00:00",
"tools": [
{
"externalReferences": [
{
"type": "build-system",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/actions"
},
{
"type": "distribution",
"url": "https://pypi.org/project/cyclonedx-python-lib/"
},
{
"type": "documentation",
"url": "https://cyclonedx-python-library.readthedocs.io/"
},
{
"type": "issue-tracker",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/issues"
},
{
"type": "license",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/LICENSE"
},
{
"type": "release-notes",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/CHANGELOG.md"
},
{
"type": "vcs",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib"
},
{
"type": "website",
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/#readme"
}
],
"name": "cyclonedx-python-lib",
"vendor": "CycloneDX",
"version": "6.4.4"
}
]
},
"serialNumber": "urn:uuid:dd68fbb0-f77c-4bb9-90cd-606dd854f301",
"version": 1,
"$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json",
"bomFormat": "CycloneDX",
"specVersion": "1.5"
}
2 changes: 1 addition & 1 deletion etc/cyclonedx.sbom.json
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@
}
],
"metadata": {
"timestamp": "2024-05-09T15:50:40.695420+00:00",
"timestamp": "2024-06-05T21:16:58.419485+00:00",
"tools": [
{
"externalReferences": [
Expand Down
29 changes: 11 additions & 18 deletions etc/garasign_dist_file.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,25 +19,18 @@ if ! command -v gpg >/dev/null; then
echo "gpg is required to verify distribution tarball signature" 1>&2
fi

creds=~/.secrets/garasign-creds.txt
artifactory_creds=~/.secrets/artifactory-creds.txt
garasign_creds=~/.secrets/garasign-creds.txt

if [[ ! -f "${creds:?}" ]]; then
echo "missing file ${creds:?}" 1>&2
exit 1
fi

# Avoid conflict/use of creds defined in the environment.
unset ARTIFACTORY_USER
unset ARTIFACTORY_PASSWORD
unset GRS_CONFIG_USER1_USERNAME
unset GRS_CONFIG_USER1_PASSWORD

. "${creds:?}"
unset ARTIFACTORY_USER ARTIFACTORY_PASSWORD
. "${artifactory_creds:?}"
: "${ARTIFACTORY_USER:?"missing ARTIFACTORY_USER in ${artifactory_creds:?}"}"
: "${ARTIFACTORY_PASSWORD:?"missing ARTIFACTORY_PASSWORD in ${artifactory_creds:?}"}"

: "${ARTIFACTORY_USER:?"missing ARTIFACTORY_USER in ${creds:?}"}"
: "${ARTIFACTORY_PASSWORD:?"missing ARTIFACTORY_PASSWORD in ${creds:?}"}"
: "${GRS_CONFIG_USER1_USERNAME:?"missing GRS_CONFIG_USER1_USERNAME in ${creds:?}"}"
: "${GRS_CONFIG_USER1_PASSWORD:?"missing GRS_CONFIG_USER1_PASSWORD in ${creds:?}"}"
unset GRS_CONFIG_USER1_USERNAME GRS_CONFIG_USER1_PASSWORD
. "${garasign_creds:?}"
: "${GRS_CONFIG_USER1_USERNAME:?"missing GRS_CONFIG_USER1_USERNAME in ${garasign_creds:?}"}"
: "${GRS_CONFIG_USER1_PASSWORD:?"missing GRS_CONFIG_USER1_PASSWORD in ${garasign_creds:?}"}"

dist_file="${1:?}"
dist_file_signed="${dist_file:?}.asc"
Expand All @@ -48,7 +41,7 @@ plugin_commands=(
gpg --yes -v --armor -o "${dist_file_signed:?}" --detach-sign "${dist_file:?}"
)
"${launcher:?}" run \
--env-file="${creds:?}" \
--env-file="${garasign_creds:?}" \
-e "PLUGIN_COMMANDS=${plugin_commands[*]:?}" \
--rm \
-v "$(pwd):$(pwd)" \
Expand Down
Loading