-
Notifications
You must be signed in to change notification settings - Fork 208
PHPC-2380: Add SBOM file and tooling to update it #1577
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -67,3 +67,6 @@ mongodb-*tgz | |
|
||
# Coverage files | ||
coverage* | ||
|
||
# temporary purls file | ||
/purls.txt | ||
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -182,63 +182,72 @@ a 1.23.1 tag also existed at the time. The bump to libmongoc 1.23.1 was left to | |
another PHPC ticket in the 1.15.0 milestone, which actually depended on the | ||
libmongoc changes therein. | ||
|
||
### Updating libmongoc and libbson | ||
### Updating bundled libraries | ||
|
||
#### Update libmongoc submodule | ||
The following steps are the same for libmongoc and libmongocrypt. When updating | ||
libmongocrypt, follow the same steps but replace `libmongoc` with | ||
`libmongocrypt`, retaining the same capitalization. The following examples | ||
always refer to libmongoc. | ||
|
||
``` | ||
$ cd src/libmongoc | ||
$ git fetch | ||
$ git checkout 1.20.0 | ||
#### Update submodule | ||
|
||
```shell | ||
cd src/libmongoc | ||
git fetch | ||
git checkout 1.20.0 | ||
``` | ||
|
||
During development, it may be necessary to temporarily point the libmongoc | ||
submodule to a commit on the developer's fork of libmongoc. For instance, the | ||
developer may be working on a PHP driver feature that depends on an unmerged | ||
pull request to libmongoc. In this case, `git remote add` can be used to add | ||
the fork before fetching and checking out the target commit. Additionally, the | ||
submodule path in | ||
[`.gitmodules`](https://github.com/mongodb/mongo-php-driver/blob/master/.gitmodules) | ||
must also be updated to refer to the fork. | ||
During development, it may be necessary to temporarily point the submodule to a | ||
commit on the developer's fork. For instance, the developer may be working on a | ||
PHP driver feature that depends on unmerged or unreleased changes. In this case, | ||
the submodule path can be updated using the `git submodules set-url` command can | ||
be used to change the URL, and `git submodules set-branch` can be used to point | ||
the submodule to a development branch: | ||
|
||
#### Ensure libmongoc version information is correct | ||
```shell | ||
git submodules set-url src/libmongoc https://github.com/<owner>/<repo>.git | ||
git submodules set-branch -b <branch> src/libmongoc | ||
``` | ||
|
||
The build process for Autotools and Windows rely on | ||
`src/LIBMONGOC_VERSION_CURRENT` to infer version information for libmongoc and | ||
libbson. This file can be regenerated using the following Makefile target: | ||
#### Ensure version information is correct | ||
|
||
``` | ||
$ make libmongoc-version-current | ||
Various build processes and tools rely on the version files to infer version | ||
information. This file can be regenerated using Makefile targets: | ||
|
||
```shell | ||
make libmongoc-version-current | ||
``` | ||
|
||
Alternatively, the `build/calc_release_version.py` script in libmongoc can be | ||
executed directly. | ||
Alternatively, the `build/calc_release_version.py` script in the submodule can | ||
be executed directly. | ||
|
||
Note: If the libmongoc submodule points to a non-release, non-master branch, the | ||
script may fail to correctly detect the version. This issue is being tracked in | ||
[CDRIVER-3315](https://jira.mongodb.org/browse/CDRIVER-3315) and can be safely | ||
ignored since this should only happen during development (any PHP driver release | ||
should point to a tagged libmongoc release). | ||
Note: If the submodule points to a non-release, non-master branch, the script | ||
may fail to correctly detect the version. This issue is being tracked in | ||
[CDRIVER-3315](https://jira.mongodb.org/browse/CDRIVER-3315) and can be safely ignored since this should only happen | ||
during development (any PHP driver release should point to a tagged submodule | ||
version). | ||
|
||
#### Update sources in build configurations | ||
|
||
The Autotools and Windows build configurations (`config.m4` and `config.w32`, | ||
respectively) define several variables (e.g. `PHP_MONGODB_MONGOC_SOURCES`) that | ||
collectively enumerate all of the the sources within the libmongoc submodule to | ||
include in a bundled build. | ||
collectively enumerate all of the sources within the submodules to include in a | ||
bundled build. | ||
|
||
These variables should each have a shell command in a preceding comment, which | ||
should be run to regenerate that particular list of source files. Each command | ||
may be run manually or `scripts/update-submodule-sources.php` may be used to | ||
update all variables. In the event that either libmongoc or libbson introduce a | ||
new source directory, that will need to be manually added (follow prior art). | ||
update all variables. In the event that a new source directory is introduced, | ||
this directory will need to be manually added following prior art. | ||
|
||
#### Update package dependencies | ||
|
||
The Autotools configuration additionally includes some `pkg-config` commands for | ||
using libmongoc and libbson as system libraries (in lieu of a bundled build). | ||
When bumping the libmongoc version, be sure to update the version check _and_ | ||
error message in the `pkg-config` blocks for both libmongoc and libbson. | ||
using libmongoc, libbson, and libmongocrypt as system libraries (in lieu of a | ||
bundled build). When bumping the bundled version, be sure to update the version | ||
check _and_ error message in the `pkg-config` blocks for the submodule being | ||
updated. When updating libmongoc, be sure to update both version checks for | ||
libmongoc and libbson. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good to call this out, even though the lines are close together. |
||
|
||
For example, the following lines might be updated for libmongoc: | ||
|
||
|
@@ -250,7 +259,7 @@ if $PKG_CONFIG libmongoc-1.0 --atleast-version 1.20.0; then | |
AC_MSG_ERROR(system libmongoc must be upgraded to version >= 1.20.0) | ||
``` | ||
|
||
#### Update tested versions in Evergreen configuration | ||
#### Update tested versions in Evergreen configuration (libmongoc only) | ||
|
||
Evergreen tests against multiple versions of libmongoc. When updating to a newer | ||
libmongoc version, make sure to update the libmongoc build tasks in `.evergreen/config/templates/build/build-libmongoc.yml` | ||
|
@@ -264,32 +273,30 @@ against two additional versions of libmongoc: | |
|
||
#### Update sources in PECL package generation script | ||
|
||
If either libmongoc or libbson introduce a new source directory, that may also | ||
If a new version of a submodule introduces a new source directory, that may also | ||
require updating the glob patterns in the `bin/prep-release.php` script to | ||
ensure new source files will be included in any generated PECL package. | ||
|
||
#### Test and commit your changes | ||
#### Update SBOM file | ||
|
||
Verify that the upgrade was successful by ensuring that the driver can compile | ||
using both the bundled sources and system libraries for libmongoc and libbson, | ||
and by ensuring that the test suite passes. Once done, commit the changes to all | ||
of the above files/paths. For example: | ||
After updating dependencies, the SBOM file needs to be updated. There is a | ||
script to automate this process: | ||
|
||
``` | ||
$ git commit -m "Bump libmongoc to 1.20.0" config.m4 config.w32 src/libmongoc src/LIBMONGOC_VERSION_CURRENT | ||
```shell | ||
./scripts/update-sbom.sh | ||
``` | ||
|
||
### Updating libmongocrypt | ||
This script will generate a temporary purl file with our dependencies, then run | ||
the internal silkbomb tool to update the SBOM. Note that you need to have docker | ||
installed in order to run this. | ||
|
||
To update libmongocrypt, the steps are similar to the above: | ||
#### Test and commit your changes | ||
|
||
``` | ||
$ cd src/libmongocrypt | ||
$ git fetch | ||
$ git checkout 1.3.0 | ||
$ make libmongocrypt-version-current | ||
``` | ||
Verify that the upgrade was successful by ensuring that the driver can compile | ||
using both the bundled sources and system libraries, and by ensuring that the | ||
test suite passes. Once done, commit the changes to all of the above | ||
files/paths. For example: | ||
|
||
Package dependencies in `config.m4` must also be updated (either manually or | ||
with `scripts/update-submodule-sources.php`), as do the sources in the PECL | ||
generation script. | ||
```shell | ||
git commit -m "Bump libmongoc to 1.20.0" config.m4 config.w32 src/libmongoc src/LIBMONGOC_VERSION_CURRENT sbom.json | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,97 @@ | ||
{ | ||
"components": [ | ||
{ | ||
"bom-ref": "pkg:github/mongodb/[email protected]", | ||
"externalReferences": [ | ||
{ | ||
"type": "distribution", | ||
"url": "https://github.com/mongodb/libmongocrypt/archive/refs/tags/1.10.0.tar.gz" | ||
}, | ||
{ | ||
"type": "website", | ||
"url": "https://github.com/mongodb/libmongocrypt/tree/1.10.0" | ||
} | ||
], | ||
"group": "mongodb", | ||
"name": "libmongocrypt", | ||
"purl": "pkg:github/mongodb/[email protected]", | ||
"type": "library", | ||
"version": "1.10.0" | ||
}, | ||
{ | ||
"bom-ref": "pkg:github/mongodb/[email protected]", | ||
"externalReferences": [ | ||
{ | ||
"type": "distribution", | ||
"url": "https://github.com/mongodb/mongo-c-driver/archive/refs/tags/1.27.2.tar.gz" | ||
}, | ||
{ | ||
"type": "website", | ||
"url": "https://github.com/mongodb/mongo-c-driver/tree/1.27.2" | ||
} | ||
], | ||
"group": "mongodb", | ||
"name": "mongo-c-driver", | ||
"purl": "pkg:github/mongodb/[email protected]", | ||
"type": "library", | ||
"version": "1.27.2" | ||
} | ||
], | ||
"dependencies": [ | ||
{ | ||
"ref": "pkg:github/mongodb/[email protected]" | ||
}, | ||
{ | ||
"ref": "pkg:github/mongodb/[email protected]" | ||
} | ||
], | ||
"metadata": { | ||
"timestamp": "2024-06-06T07:13:52.679415+00:00", | ||
"tools": [ | ||
{ | ||
"externalReferences": [ | ||
{ | ||
"type": "build-system", | ||
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/actions" | ||
}, | ||
{ | ||
"type": "distribution", | ||
"url": "https://pypi.org/project/cyclonedx-python-lib/" | ||
}, | ||
{ | ||
"type": "documentation", | ||
"url": "https://cyclonedx-python-library.readthedocs.io/" | ||
}, | ||
{ | ||
"type": "issue-tracker", | ||
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/issues" | ||
}, | ||
{ | ||
"type": "license", | ||
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/LICENSE" | ||
}, | ||
{ | ||
"type": "release-notes", | ||
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/blob/main/CHANGELOG.md" | ||
}, | ||
{ | ||
"type": "vcs", | ||
"url": "https://github.com/CycloneDX/cyclonedx-python-lib" | ||
}, | ||
{ | ||
"type": "website", | ||
"url": "https://github.com/CycloneDX/cyclonedx-python-lib/#readme" | ||
} | ||
], | ||
"name": "cyclonedx-python-lib", | ||
"vendor": "CycloneDX", | ||
"version": "6.4.4" | ||
} | ||
] | ||
}, | ||
"serialNumber": "urn:uuid:acb30d08-ee47-4ff0-b301-d66ef1f54082", | ||
"version": 1, | ||
"$schema": "http://cyclonedx.org/schema/bom-1.5.schema.json", | ||
"bomFormat": "CycloneDX", | ||
"specVersion": "1.5" | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,19 @@ | ||
#!/usr/bin/env bash | ||
|
||
SCRIPT_DIR=$(dirname ${BASH_SOURCE[0]}) | ||
ROOT_DIR=$(realpath "${SCRIPT_DIR}/../") | ||
PURLS_FILE="${ROOT_DIR}/purls.txt" | ||
|
||
LIBMONGOC_VERSION=$(cat ${ROOT_DIR}/src/LIBMONGOC_VERSION_CURRENT | tr -d '[:space:]') | ||
LIBMONGOCRYPT_VERSION=$(cat ${ROOT_DIR}/src/LIBMONGOCRYPT_VERSION_CURRENT | tr -d '[:space:]') | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I assume we don't need to call out that these files must be updated since the SBOM generation step follows the "Ensure version information is correct" step, which updates these. |
||
|
||
# Generate purls file from stored versions | ||
echo "pkg:github/mongodb/mongo-c-driver@${LIBMONGOC_VERSION}" > $PURLS_FILE | ||
echo "pkg:github/mongodb/libmongocrypt@${LIBMONGOCRYPT_VERSION}" >> $PURLS_FILE | ||
|
||
# Use silkbomb to update the sbom.json file | ||
docker run --platform="linux/amd64" -it --rm -v ${ROOT_DIR}:/pwd \ | ||
artifactory.corp.mongodb.com/release-tools-container-registry-public-local/silkbomb:1.0 \ | ||
update --sbom-in /pwd/sbom.json --purls /pwd/purls.txt --sbom-out /pwd/sbom.json | ||
|
||
rm $PURLS_FILE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Noted that you only want to ignore this in the project root directory.