[CDRIVER-4637] A config settings command #1289

vector-of-bool · 2023-05-31T19:54:57Z

This changeset is partial work towards CDRIVER-4637. It introduces a CMake module MongoSettings.cmake which defines CMake utilities for defining build settings in a reliable, uniform, and expressive manner.

Originally, this changeset was going to also complete CDRIVER-4637 and remove the AUTO build settings that are used throughout the project. However, this proved to be an enormous task as many of our CI configurations implicitly relied AUTO to make things "just work" regardless of the ambient build environment (this is exactly the thing we are purposefully trying to remove via CDRIVER-4637). For this reason, updating any individual build setting away from AUTO is substantial effort to coordinate with the CI executors as to when and where certain features are available.

Setting Design

The mongo_setting() command looks like the following:

mongo_setting(
    <name> <doc>
    [TYPE <BOOL|PATH|FILEPATH|STRING>]
    [DEFAULT [[DEVEL|AUDIT] [VALUE <value> | EVAL <code>]] ...]
    [OPTIONS [<opts> ...]]
    [VALIDATE [CODE <code>]]
    [ADVANCED] [ALLOW_OTHER_VALUES]
)

The full explanation of all arguments is available in MongoSettings.cmake. It basically works like this:

<name>, <doc>, and TYPE correspond to the parameters to set() in set(<name> [...] CACHE <TYPE> <doc>)
The DEFAULT argument takes one of two forms: VALUE and EVAL
1. For VALUE <value>, the given <value> is set as the default value. Simple.
2. For EVAL <code>, the given <code> is executed in order to determine the default value. This is used where the default value should depend on certain conditions about the system, such as the operating system and host architecture. Importantly, it SHOULD NOT depend on things like the result of a find_package() or ambient system state. This is designed to replace uses of AUTO to toggle operating-system-specific settings. Currently, it is only used in one location, to turn on ENABLE_SHM_COUNTERS in the same conditions under which AUTO would have enabled this feature (this is the only feature that I have found does not generate trouble when removing AUTO).
The OPTIONS argument specifies a list of possible values that may be taken by the build setting. This also sets the STRINGS property on the cache variable. If provided, mongo_setting will validate that the build setting is one of these values unless ALLOW_OTHER_VALUES is also given.
ADVANCED will mark the cache variable as advanced.
VALIDATE CODE <code> will execute <code> after the cache value has been set. This code should issue an error/warning if the value is incompatible with another setting or with the toolchain/platform.
The DEVEL and AUDIT toggles to DEFAULT allow different default default values to be set when certain environment variables are present. This isn't strictly related to CDRIVER-4637, but is a convenience for developers and CI.

Additionally, there is a mongo_bool_setting() command that defines a build setting of type BOOL with an implicit default of TRUE.

CMake Version Updates

This changeset requires a more recent (but still relatively old) CMake version. Several build tasks were still using prior find-cmake logic. Those have been updated to use find-cmake-latest.

It remains that the build-with-toolchain uses an old CMake version that conflicts with the newer CMake required. Addressing this is an open problem.

MinGW Changes

The MinGW build tasks invoked a .bat script that hardcoded a path to CMake and GCC. The CMake that it used is too old to work with these changes, and batch is undoubtedly the worse scripting language ever conceived and widely adopted.

Since we now have the technology, the MinGW task has been modified to stay within Cygwin Bash and invoke the CMake command correctly to build with MinGW GCC. This removes the hardcoding of the (ancient 4.9) MinGW GCC version, and CMake now finds a more modern GCC 8 available on the PATH. Ideally, we would not rely on an arbitrary GCC being available for this process, so I opened CDRIVER-4652 to address this, hopefully sooner than later.

Almost all setting values have remained the same, and most setting handling logic has remained unmodified, but this will serve as a basis for better configuration settings handling in the future tasks for CDRIVER-4637.

Using `message()` without a mode argument will write the message as a NOTICE to stderr. We want to use STATUS as the default, which goes to stdout. Systems which colorize stderr output would seemingly-randomly color certain lines where the mode argument was missing.

Several build tasks still relied on older find-cmake logic. With the new CMake changes, a more recent CMake version is required. These tasks and scripts have been updated to use the same logic to find the same CMake version as is used in the main build tasks.

Batch scripting is fraught with peril and will almost never do what one expects. Any other scripting language is preferable. Since we have Bash available and the tools to properly convert paths, just use Bash to do this work.

kevinAlbs

The added logs showing value changes seem very helpful.

Noted a slight concerned about raising the minimum cmake for users on older Debian versions that have older packaged cmake.

Noted a concern about changing the accepted values in options. If accepted option values are changed, I prefer the old value result in a configuration error. I want to prevent silently changing the build for existing build scripts.

build/cmake/MongoSettings.cmake

kevinAlbs · 2023-06-02T14:29:53Z

CMakeLists.txt

-   "Turn on extra alignment of libbson types. Set to ON/OFF, default ON.\
- Required for the 1.0 ABI but better disabled."
-   ON
+cmake_minimum_required (VERSION 3.15)


Debian 10 (Buster) appears to have cmake 3.13 packaged. Testing with Debian 9.2 on a spawn host appears to have cmake 3.7 packaged.

I expect that may be a hurdle for users wanting to compile from source on older Debian versions. Though users can download and install a newer version of cmake separately. I suggest adding to the NEWS file to alert users wanting to upgrade:

* The minimum required cmake version has increased to 3.15.

Atlas Usage Statistics for C: 2022-12-14 includes percentage of users on Debian 9 and 10. (Regenerating this data depends on ANALYTICS-6106)

Would decreasing the minimum to cmake 3.7 require significant workarounds in this PR?

So, as an FYI, there are actually backports of newer CMake packages available from the (semi-)official backports repo. For Debian 9:

root@manzanillo:~# cat /etc/debian_version 9.13 root@manzanillo:~# apt-cache policy cmake cmake: Installed: (none) Candidate: 3.7.2-1 Version table: 3.16.3-1~bpo9+1 100 100 http://archive.debian.org/debian stretch-backports/main amd64 Packages 3.7.2-1 500 500 http://archive.debian.org/debian stretch/main amd64 Packages

And for Debian 10:

root@manzanillo:~# cat /etc/debian_version 10.13 root@manzanillo:~# apt-cache policy cmake cmake: Installed: (none) Candidate: 3.13.4-1 Version table: 3.18.4-2+deb11u1~bpo10+1 100 100 http://ftp.us.debian.org/debian buster-backports/main amd64 Packages 3.13.4-1 500 500 http://ftp.us.debian.org/debian buster/main amd64 Packages

It might be sufficient to add a note along the lines of "if you are using Debian 9 or 10 then you will need to install cmake from the (buster|stretch)-backports source". Keep in mind that Debian 9 ended its LTS window about a year ago and for Debian 10 the LTS window goes for about one year (until June 2024). There is a commercial product that continues to offer support for Debian releases for 5 years after LTS ends, but I think it is safe to assume that there is little to no overlap between users of 5-10 year old Debian releases and users of the latest Mongo DB C driver.

Thank you for the additional information. I am OK with bumping the minimum CMake.

I recommend updating the NEWS.

It remains that the build-with-toolchain uses an old CMake version that conflicts with the newer CMake required. Addressing this is an open problem.

I expect the toolchain will need to be updated. The toolchain is built from this repo: https://github.com/10gen/mongo-c-toolchain as part of this Evergreen project: https://spruce.mongodb.com/commits/mongo-c-driver-toolchain

Opened this: https://github.com/10gen/mongo-c-toolchain/pull/2

CMakeLists.txt

kevinAlbs · 2023-06-02T15:56:51Z

build/cmake/MongoSettings.cmake

+    if(setting_TYPE STREQUAL "BOOL"
+        AND NOT setting_ALLOW_OTHER_VALUES
+        AND NOT "${${setting_NAME}}" MATCHES "^(TRUE|FALSE|ON|OFF|YES|NO|0|1)$")
+        message(WARNING "The value of ${setting_NAME}=“${${setting_NAME}}” is not a regular boolean value")


Suggested change

message(WARNING "The value of ${setting_NAME}=“${${setting_NAME}}” is not a regular boolean value")

message(FATAL_ERROR "The value of ${setting_NAME}=“${${setting_NAME}}” is not a regular boolean value. Set to TRUE or FALSE")

I'm inclined to leave this. The ENABLE_BSON option is now handled separately.

.evergreen/scripts/compile-unix.sh

build/cmake/MongoSettings.cmake

src/libmongoc/CMakeLists.txt

rcsanchez97

I have made a few comments for your consideration, but overall this looks good. Approving now so that if I am not able to review again in the next few days I am not holding up a merge.

LGTM

rcsanchez97 · 2023-06-01T18:45:04Z

.evergreen/config_generator/components/make_release_archive.py

@@ -23,6 +23,7 @@ class ReleaseArchive(Function):
            script='''\
                set -o errexit
                bash tools/poetry.sh install --with=docs
+                export distro_id=${distro_id}  # Needed by find-cmake-latest.sh


In one of my PRs, Ezra pointed out that the you can use include_expansions_in_env: ["distro_id"] under params in place of an explicit export.

rcsanchez97 · 2023-06-01T18:45:40Z

.evergreen/generated_configs/functions.yml

@@ -284,6 +284,7 @@ functions:
          - |
            set -o errexit
            bash tools/poetry.sh install --with=docs
+            export distro_id=${distro_id}  # Needed by find-cmake-latest.sh


In one of my PRs, Ezra pointed out that the you can use include_expansions_in_env: ["distro_id"] under params in place of an explicit export.

rcsanchez97 · 2023-06-06T16:46:52Z

CMakeLists.txt

-   "Turn on extra alignment of libbson types. Set to ON/OFF, default ON.\
- Required for the 1.0 ABI but better disabled."
-   ON
+cmake_minimum_required (VERSION 3.15)


So, as an FYI, there are actually backports of newer CMake packages available from the (semi-)official backports repo. For Debian 9:

root@manzanillo:~# cat /etc/debian_version 9.13 root@manzanillo:~# apt-cache policy cmake cmake: Installed: (none) Candidate: 3.7.2-1 Version table: 3.16.3-1~bpo9+1 100 100 http://archive.debian.org/debian stretch-backports/main amd64 Packages 3.7.2-1 500 500 http://archive.debian.org/debian stretch/main amd64 Packages

And for Debian 10:

root@manzanillo:~# cat /etc/debian_version 10.13 root@manzanillo:~# apt-cache policy cmake cmake: Installed: (none) Candidate: 3.13.4-1 Version table: 3.18.4-2+deb11u1~bpo10+1 100 100 http://ftp.us.debian.org/debian buster-backports/main amd64 Packages 3.13.4-1 500 500 http://ftp.us.debian.org/debian buster/main amd64 Packages

It might be sufficient to add a note along the lines of "if you are using Debian 9 or 10 then you will need to install cmake from the (buster|stretch)-backports source". Keep in mind that Debian 9 ended its LTS window about a year ago and for Debian 10 the LTS window goes for about one year (until June 2024). There is a commercial product that continues to offer support for Debian releases for 5 years after LTS ends, but I think it is safe to assume that there is little to no overlap between users of 5-10 year old Debian releases and users of the latest Mongo DB C driver.

rcsanchez97 · 2023-06-06T16:52:59Z

CMakeLists.txt

+
+# Subcomponents:
+mongo_bool_setting(ENABLE_MONGOC "Enable the build of the mongo-c-driver libraries")
+mongo_bool_setting(ENABLE_BSON "Enable the build of the C BSON handling libraries")


Personally as a package maintainer, I'd prefer the fatal error. It is entirely likely that I would never notice the warning message when going through the process of packaging a new release, but I would definitely notice the error.

- ENABLE_BSON=ON did the same as ENABLE_BSON=AUTO - There is no "ENABLE_BSON=OFF" since then there would be nothing to build. - USE_SYSTEM_LIBBSON=ON replaces ENABLE_BSON=SYSTEM. - To alleviate downstreams from needing to check versions before configuring, allow ENABLE_BSON=SYSTEM to be specified if USE_SYSTEM_LIBBSON=ON is also set, thus allowing one option pair to build both before and after this revision. - If ENABLE_BSON=SYSTEM is set but USE_SYSTEM_LIBBSON is not TRUE, then downstream will want to update their scripts to build with the new option rather than miscompile without being noticed.

kevinAlbs

LGTM with open comments addressed.

kevinAlbs · 2023-06-06T19:48:28Z

CMakeLists.txt

+
+# Deprecated option "ENABLE_BSON" is gone, but old caches may rely on SYSTEM to
+# disable the libbson build. Allow ENABLE_BSON=SYSTEM only if USE_SYSTEM_LIBBSON
+# is also true, to allow both ENABLE_BSON=SYSTEM and USE_SYSTEM_LIBBSON=TRUE to


Allowing both options seems helpful. Consider adding a bullet to the unreleased section of the NEWS to warn users of the change:

The ENABLE_BSON configuration option has been removed. Downstream build scripts using ENABLE_BSON=SYSTEM should be updated to use USE_SYSTEM_LIBBSON=ON. If the build script needs to support multiple versions of the C driver, add both ENABLE_BSON=SYSTEM and USE_SYSTEM_LIBBSON=TRUE.

kevinAlbs · 2023-06-06T19:52:00Z

CMakeLists.txt

-   "Turn on extra alignment of libbson types. Set to ON/OFF, default ON.\
- Required for the 1.0 ABI but better disabled."
-   ON
+cmake_minimum_required (VERSION 3.15)


Thank you for the additional information. I am OK with bumping the minimum CMake.

I recommend updating the NEWS.

It remains that the build-with-toolchain uses an old CMake version that conflicts with the newer CMake required. Addressing this is an open problem.

I expect the toolchain will need to be updated. The toolchain is built from this repo: https://github.com/10gen/mongo-c-toolchain as part of this Evergreen project: https://spruce.mongodb.com/commits/mongo-c-driver-toolchain

eramongodb

Great work! The DEVEL options pattern is very neat. For the most part, LGTM, but there are several suggestion and questions I would like to be addressed.

.evergreen/config_generator/components/make_release_archive.py

.evergreen/scripts/compile-unix.sh

CMakeLists.txt

build/cmake/MongoSettings.cmake

rcsanchez97 · 2023-06-07T13:40:59Z

OK. So, I did some research and I think you should go a little further and also remove the `USE_SYSTEM_LIBBSON` option you introduced. The history here is that prior to 1.10.0 libbson and the C driver were two separate projects. A user could clone libbson and build/install it only. If you wanted the C driver, you had to first get/build/install libbson and then get/build/install the C driver. When the two projects were merged into a single source repository and a single CMake project, we introduced the `ENABLE_BSON` option to allow the user to either build libbson and the C driver together in one step, or separately in two steps (by using the `SYSTEM` value to tell the C driver build in the second step to go look for the libbson already present on the system and the `BUNDLED` value to tell it to look for the libbson source in a sub-directory in a one-step build). The support for a two-step build, IIRC, was to allow users to keep existing two-step build workflows intact. It also had to do with the libbson repo being initially merged into the C driver repo as a top-level project and then only later being made into a sub-project (from the CMake perspective). However, I can't imagine that any of this ia anything that we need to worry about that any longer (it has been more than 5 years, after all). You can get a more complete picture of the evolution by looking at CDRIVER-2505 and the commits associated with it. Keeping `ENABLE_MONGOC` around makes a lot of sense, since it is possible that a user wants only libbson (either because libbson is the only requirement for a given project, or it is the only required dependency like for libmongocrypt). In effect, that makes `ENABLE_MONGOC` necessary. However, there is no way to build the C driver /without/ libbson and these days it seems like a vanishingly small possibility that someone would want to build only the C driver against an already existing libbson installation. The possibility is so small, in my opinion, that is essesntially zero and not worth going to the effort to keep around (even in the form of `USE_SYSTEM_LIBBSON`) if you are already tweaking the option anyways. Better to just drop it altogether. In summary, there are basically two use cases that relate to the potential need for `ENABLE_BSON` and/or `USE_SYSTEM_LIBBSON`. 1. A user wants to build only libbson without libmongoc 2. A user wants to build only libmongoc without libbson (i.e., using a libbson previously built and installed on the system) The first is accomplished with `ENABLE_MONGOC` and the second hasn't been a thing for probably 5+ years and so keeping an option around for it adds complexity and maintenance burden for no discernable benefit.

vector-of-bool · 2023-06-07T21:24:35Z

@rcsanchez97 How would this work for packagers that only want to build/install the libmongoc component against a previously-packaged libbson? e.g. Using ENABLE_BSON=SYSTEM is how it is currently done in vcpkg Refer.

I think there is more simplification to be done in this regard, but I think it may be for a later date. (Possible within the same scope of CDRIVER-4546).

eramongodb

I'd like to wait for https://github.com/10gen/mongo-c-toolchain/pull/2 to be merged so we can verify the link-with-* tasks pass with the proposed changes to avoid interfering with patch builds. Otherwise, LGTM.

Concerning USE_SYSTEM_LIBBSON, if we can encourage package maintainers to simplify their handling of libbson/libmongoc to avoid dealing with the system libbson situation (e.g. by making the libmongoc package conflict with the libbson package due to unconditionally providing its own libbson), I think I would prefer going down that route. If we cannot (e.g. too disruptive, or legitimate use cases for system/external libbson dependencies), then this option should remain.

.evergreen/config_generator/etc/utils.py

CMakeLists.txt

eramongodb · 2023-06-08T15:25:52Z

CMakeLists.txt

+)
+
+mongo_bool_setting(
+   ENABLE_EXTRA_ALIGNMENT


Suggest relocating below alongside ENABLE_AUTOMATIC_INIT_AND_CLEANUP under "Deprecated options:".

The toolchain CMake is incompatible with OpenSSL 1.1.0, so having it on the library path breaks CMake. We don't need to modify LD_LIBRARY_PATH to cause CMake to use the OpenSSL that we want. It is sufficient to prepend the SSL root to the CMAKE_PREFIX_PATH.

rcsanchez97 · 2023-06-10T16:52:50Z

@rcsanchez97 How would this work for packagers that only want to build/install the libmongoc component against a previously-packaged libbson? e.g. Using ENABLE_BSON=SYSTEM is how it is currently done in vcpkg Refer.

So, it looks like vcpkg handles things a bit differently. Their approach would still require the USE_SYSTEM_LIBBSON option. Their choice to build libbson in one place and then the C driver in another place (and then having that C driver build depend on the previously built libbson) is clearly a current instance of the two step build I described.

I think there is more simplification to be done in this regard, but I think it may be for a later date. (Possible within the same scope of CDRIVER-4546).

Cool. I'm looking forward to that.

* New config settings code and defaults. Almost all setting values have remained the same, and most setting handling logic has remained unmodified, but this will serve as a basis for better configuration settings handling in the future tasks for CDRIVER-4637. * Aside: message() cleanup Using `message()` without a mode argument will write the message as a NOTICE to stderr. We want to use STATUS as the default, which goes to stdout. Systems which colorize stderr output would seemingly-randomly color certain lines where the mode argument was missing. * Use new CMake in more build tasks Several build tasks still relied on older find-cmake logic. With the new CMake changes, a more recent CMake version is required. These tasks and scripts have been updated to use the same logic to find the same CMake version as is used in the main build tasks. * Don't use Batch scripts to compile for MinGW Batch scripting is fraught with peril and will almost never do what one expects. Any other scripting language is preferable. Since we have Bash available and the tools to properly convert paths, just use Bash to do this work. * Redundant ENABLE_SHM_COUNTERS logic * Update the "Ask for Help" section * No longer applicable SASL checks * Remove ENABLE_BSON, as its purpose is confusing. - ENABLE_BSON=ON did the same as ENABLE_BSON=AUTO - There is no "ENABLE_BSON=OFF" since then there would be nothing to build. - USE_SYSTEM_LIBBSON=ON replaces ENABLE_BSON=SYSTEM. - To alleviate downstreams from needing to check versions before configuring, allow ENABLE_BSON=SYSTEM to be specified if USE_SYSTEM_LIBBSON=ON is also set, thus allowing one option pair to build both before and after this revision. - If ENABLE_BSON=SYSTEM is set but USE_SYSTEM_LIBBSON is not TRUE, then downstream will want to update their scripts to build with the new option rather than miscompile without being noticed. * New toolchain for the build * Don't inject the toolchain SSL into LD_LIBRARY_PATH The toolchain CMake is incompatible with OpenSSL 1.1.0, so having it on the library path breaks CMake. We don't need to modify LD_LIBRARY_PATH to cause CMake to use the OpenSSL that we want. It is sufficient to prepend the SSL root to the CMAKE_PREFIX_PATH. * Coverage link options are usage requirements * More Bash, less Sh * Don't depend on distro_id

vector-of-bool added 4 commits May 10, 2023 19:10

New config settings code and defaults.

b131314

Almost all setting values have remained the same, and most setting handling logic has remained unmodified, but this will serve as a basis for better configuration settings handling in the future tasks for CDRIVER-4637.

Don't use Batch scripts to compile for MinGW

5979a33

Batch scripting is fraught with peril and will almost never do what one expects. Any other scripting language is preferable. Since we have Bash available and the tools to properly convert paths, just use Bash to do this work.

vector-of-bool requested review from rcsanchez97, kevinAlbs and eramongodb May 31, 2023 19:54

vector-of-bool marked this pull request as ready for review May 31, 2023 19:55

Redundant ENABLE_SHM_COUNTERS logic

947b91e

kevinAlbs reviewed Jun 2, 2023

View reviewed changes

vector-of-bool added 4 commits June 2, 2023 22:19

Comment and message tweaks

8d3854f

Update the "Ask for Help" section

c4f81a8

Remove AUDIT defaults

026a86a

Don't enable tracing as a default

3074a0b

vector-of-bool force-pushed the CDRIVER-4637-settings-command branch from 900a2a2 to 3074a0b Compare June 5, 2023 18:00

vector-of-bool requested a review from kevinAlbs June 5, 2023 18:02

vector-of-bool added 4 commits June 5, 2023 18:08

Tweak default of ENABLE_AUTOMATIC_INIT_AND_CLEANUP

5f02e41

More thorough boolean checks

af82494

No longer applicable SASL checks

c682aac

Merge branch 'master' into CDRIVER-4637-settings-command

d297655

rcsanchez97 approved these changes Jun 6, 2023

View reviewed changes

vector-of-bool added 2 commits June 6, 2023 19:22

EVG has an option for that

988bca3

kevinAlbs approved these changes Jun 6, 2023

View reviewed changes

eramongodb reviewed Jun 6, 2023

View reviewed changes

Many PR comments and wording tweaks. Remove some unneeded bits

31a0bf9

vector-of-bool requested a review from eramongodb June 7, 2023 22:18

vector-of-bool mentioned this pull request Jun 8, 2023

CDRIVER-4637 Remove ENABLE_SRV=AUTO, clean up SRV URI support #1301

Merged

eramongodb approved these changes Jun 8, 2023

View reviewed changes

vector-of-bool added 10 commits June 8, 2023 18:35

New toolchain for the build

04ec48e

Move that to the deprecated section

b9f9f69

Merge branch 'master' into CDRIVER-4637-settings-command

1116cd8

Fix error with CMake genex parsing

da33f9c

More CMake updates

6a425ba

Coverage link options are usage requirements

70cf917

More Bash, less Sh

d042041

More bash, more CMake updates

d47bd59

Don't depend on distro_id

4e47388

vector-of-bool added 2 commits June 12, 2023 19:43

Fix macOS and Windows

d82678a

Fix setup scripts

16bbf80

vector-of-bool merged commit 3165a35 into mongodb:master Jun 13, 2023

kevinAlbs mentioned this pull request Jun 22, 2023

CDRIVER-4119 Replace ICU SASLprep with internal implementation #1317

Merged

	message(WARNING "The value of ${setting_NAME}=“${${setting_NAME}}” is not a regular boolean value")
	message(FATAL_ERROR "The value of ${setting_NAME}=“${${setting_NAME}}” is not a regular boolean value. Set to TRUE or FALSE")

[CDRIVER-4637] A config settings command #1289

[CDRIVER-4637] A config settings command #1289

Uh oh!

Conversation

vector-of-bool commented May 31, 2023

Setting Design

CMake Version Updates

MinGW Changes

Uh oh!

kevinAlbs left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rcsanchez97 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

kevinAlbs left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

eramongodb left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rcsanchez97 commented Jun 7, 2023 via email

Uh oh!

vector-of-bool commented Jun 7, 2023

Uh oh!

eramongodb left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rcsanchez97 commented Jun 10, 2023

Uh oh!

Uh oh!

eramongodb left a comment •

edited

Loading