CDRIVER-4199 comment option for CRUD methods and change streams

Sync unified change stream and CRUD tests with mongodb/specifications@b8371ea

Comment option for bulkWrite, insert, update, delete, findAndModify, aggregate, and change stream operations. Attach cursor comment to getMore for MongoDB 4.4+

Surface entity_map_match error if expectResult fails. The entity_map_match error contains relevant information about the failed match (e.g. field path). This is consistent with error reporting for error.expectResult later in the function.

Update server doc links for database, collection, index enumeration. Also moves the retryable reads include for listDatabases to be consistent with other pages.

Use BSON_ASSERT_PARAM for non-null parameter assertions in mongoc-bulk-operation.c.

Co-authored-by: Kevin Albertson <[email protected]>

Author: jmikola

build/generate-opts.py

@@ -156,11 +156,24 @@ def __init__(self, items, **defaults):
     'help': 'A BSON document consisting of any number of parameter names, each followed by definitions of constants in the MQL Aggregate Expression language.'
 })
 
+comment_option_since_4_4 = ('comment', {
+    'type': 'bson_value_t',
+    'convert': '_mongoc_convert_bson_value_t',
+    'help': 'A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.'
+})
+
+comment_option_string_pre_4_4 = ('comment', {
+    'type': 'bson_value_t',
+    'convert': '_mongoc_convert_bson_value_t',
+    'help': 'A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Only string values are supported prior to MongoDB 4.4.'
+})
+
 opts_structs = OrderedDict([
     ('mongoc_crud_opts_t', Shared([
         write_concern_option,
         session_option,
         validate_option,
+        comment_option_since_4_4,
     ])),
 
     ('mongoc_update_opts_t', Shared([
@@ -217,6 +230,7 @@ def __init__(self, items, **defaults):
         ordered_option,
         session_option,
         let_option,
+        comment_option_since_4_4,
     ], allow_extra=False, ordered='true')),
 
     ('mongoc_bulk_insert_opts_t', Struct([
@@ -279,6 +293,7 @@ def __init__(self, items, **defaults):
         ('startAtOperationTime', {'type': 'timestamp', 'help': 'A ``Timestamp``. The change stream only provides changes that occurred at or after the specified timestamp. Any command run against the server will return an operation time that can be used here. This option is mutually exclusive with ``resumeAfter`` and ``startAfter``.'}),
         ('maxAwaitTimeMS', {'type': 'int64_t', 'convert': '_mongoc_convert_int64_positive', 'help': 'An ``int64`` representing the maximum amount of time a call to :symbol:`mongoc_change_stream_next` will block waiting for data'}),
         ('fullDocument', {'type': 'utf8', 'help': 'A UTF-8 string. Set this option to "updateLookup" to direct the change stream cursor to lookup the most current majority-committed version of the document associated to an update change stream event.'}),
+        comment_option_string_pre_4_4,
     ], fullDocument="default")),
 
     ('mongoc_create_index_opts_t', Struct([
@@ -330,13 +345,15 @@ def __init__(self, items, **defaults):
         server_option,
         ('batchSize', {'type': 'int32_t', 'help': 'An ``int32`` representing number of documents requested to be returned on each call to :symbol:`mongoc_cursor_next`', 'check_set': True}),
         let_option,
+        comment_option_string_pre_4_4,
     ])),
 
     ('mongoc_find_and_modify_appended_opts_t', Struct([
         write_concern_option,
         session_option,
         hint_option,
         let_option,
+        comment_option_since_4_4,
     ], opts_name='extra'))
 ])
 

src/libmongoc/doc/includes/aggregate-opts.txt

@@ -13,3 +13,4 @@
 * ``serverId``: To target a specific server, include an int32 "serverId" field. Obtain the id by calling :symbol:`mongoc_client_select_server`, then :symbol:`mongoc_server_description_id` on its return value.
 * ``batchSize``: An ``int32`` representing number of documents requested to be returned on each call to :symbol:`mongoc_cursor_next`
 * ``let``: A BSON document consisting of any number of parameter names, each followed by definitions of constants in the MQL Aggregate Expression language.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Only string values are supported prior to MongoDB 4.4.

src/libmongoc/doc/includes/bulk-opts.txt

@@ -9,3 +9,4 @@
 * ``ordered``: set to ``false`` to attempt to insert all documents, continuing after errors.
 * ``sessionId``: First, construct a :symbol:`mongoc_client_session_t` with :symbol:`mongoc_client_start_session`. You can begin a transaction with :symbol:`mongoc_client_session_start_transaction`, optionally with a :symbol:`mongoc_transaction_opt_t` that overrides the options inherited from |opts-source|, and use :symbol:`mongoc_client_session_append` to add the session to ``opts``. See the example code for :symbol:`mongoc_client_session_t`.
 * ``let``: A BSON document consisting of any number of parameter names, each followed by definitions of constants in the MQL Aggregate Expression language.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.

src/libmongoc/doc/includes/change-stream-opts.txt

@@ -11,3 +11,4 @@
 * ``startAtOperationTime``: A ``Timestamp``. The change stream only provides changes that occurred at or after the specified timestamp. Any command run against the server will return an operation time that can be used here. This option is mutually exclusive with ``resumeAfter`` and ``startAfter``.
 * ``maxAwaitTimeMS``: An ``int64`` representing the maximum amount of time a call to :symbol:`mongoc_change_stream_next` will block waiting for data
 * ``fullDocument``: A UTF-8 string. Set this option to "updateLookup" to direct the change stream cursor to lookup the most current majority-committed version of the document associated to an update change stream event.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Only string values are supported prior to MongoDB 4.4.

src/libmongoc/doc/includes/delete-many-opts.txt

@@ -8,6 +8,7 @@
 * ``writeConcern``: Construct a :symbol:`mongoc_write_concern_t` and use :symbol:`mongoc_write_concern_append` to add the write concern to ``opts``. See the example code for :symbol:`mongoc_client_write_command_with_opts`.
 * ``sessionId``: First, construct a :symbol:`mongoc_client_session_t` with :symbol:`mongoc_client_start_session`. You can begin a transaction with :symbol:`mongoc_client_session_start_transaction`, optionally with a :symbol:`mongoc_transaction_opt_t` that overrides the options inherited from |opts-source|, and use :symbol:`mongoc_client_session_append` to add the session to ``opts``. See the example code for :symbol:`mongoc_client_session_t`.
 * ``validate``: Construct a bitwise-or of all desired :symbol:`bson_validate_flags_t <bson_validate_with_error>`. Set to ``false`` to skip client-side validation of the provided BSON documents.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.
 * ``collation``: Configure textual comparisons. See :ref:`Setting Collation Order <setting_collation_order>`, and `the MongoDB Manual entry on Collation <https://docs.mongodb.com/manual/reference/collation/>`_. Collation requires MongoDB 3.2 or later, otherwise an error is returned.
 * ``hint``: A document or string that specifies the index to use to support the query predicate.
 * ``let``: A BSON document consisting of any number of parameter names, each followed by definitions of constants in the MQL Aggregate Expression language.

src/libmongoc/doc/includes/delete-one-opts.txt

@@ -8,6 +8,7 @@
 * ``writeConcern``: Construct a :symbol:`mongoc_write_concern_t` and use :symbol:`mongoc_write_concern_append` to add the write concern to ``opts``. See the example code for :symbol:`mongoc_client_write_command_with_opts`.
 * ``sessionId``: First, construct a :symbol:`mongoc_client_session_t` with :symbol:`mongoc_client_start_session`. You can begin a transaction with :symbol:`mongoc_client_session_start_transaction`, optionally with a :symbol:`mongoc_transaction_opt_t` that overrides the options inherited from |opts-source|, and use :symbol:`mongoc_client_session_append` to add the session to ``opts``. See the example code for :symbol:`mongoc_client_session_t`.
 * ``validate``: Construct a bitwise-or of all desired :symbol:`bson_validate_flags_t <bson_validate_with_error>`. Set to ``false`` to skip client-side validation of the provided BSON documents.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.
 * ``collation``: Configure textual comparisons. See :ref:`Setting Collation Order <setting_collation_order>`, and `the MongoDB Manual entry on Collation <https://docs.mongodb.com/manual/reference/collation/>`_. Collation requires MongoDB 3.2 or later, otherwise an error is returned.
 * ``hint``: A document or string that specifies the index to use to support the query predicate.
 * ``let``: A BSON document consisting of any number of parameter names, each followed by definitions of constants in the MQL Aggregate Expression language.

src/libmongoc/doc/includes/find-and-modify-appended-opts.txt

@@ -9,3 +9,4 @@
 * ``sessionId``: First, construct a :symbol:`mongoc_client_session_t` with :symbol:`mongoc_client_start_session`. You can begin a transaction with :symbol:`mongoc_client_session_start_transaction`, optionally with a :symbol:`mongoc_transaction_opt_t` that overrides the options inherited from |opts-source|, and use :symbol:`mongoc_client_session_append` to add the session to ``opts``. See the example code for :symbol:`mongoc_client_session_t`.
 * ``hint``: A document or string that specifies the index to use to support the query predicate.
 * ``let``: A BSON document consisting of any number of parameter names, each followed by definitions of constants in the MQL Aggregate Expression language.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.

src/libmongoc/doc/includes/insert-many-opts.txt

@@ -8,5 +8,6 @@
 * ``writeConcern``: Construct a :symbol:`mongoc_write_concern_t` and use :symbol:`mongoc_write_concern_append` to add the write concern to ``opts``. See the example code for :symbol:`mongoc_client_write_command_with_opts`.
 * ``sessionId``: First, construct a :symbol:`mongoc_client_session_t` with :symbol:`mongoc_client_start_session`. You can begin a transaction with :symbol:`mongoc_client_session_start_transaction`, optionally with a :symbol:`mongoc_transaction_opt_t` that overrides the options inherited from |opts-source|, and use :symbol:`mongoc_client_session_append` to add the session to ``opts``. See the example code for :symbol:`mongoc_client_session_t`.
 * ``validate``: Construct a bitwise-or of all desired :symbol:`bson_validate_flags_t <bson_validate_with_error>`. Set to ``false`` to skip client-side validation of the provided BSON documents.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.
 * ``ordered``: set to ``false`` to attempt to insert all documents, continuing after errors.
 * ``bypassDocumentValidation``: Set to ``true`` to skip server-side schema validation of the provided BSON documents.

src/libmongoc/doc/includes/insert-one-opts.txt

@@ -8,4 +8,5 @@
 * ``writeConcern``: Construct a :symbol:`mongoc_write_concern_t` and use :symbol:`mongoc_write_concern_append` to add the write concern to ``opts``. See the example code for :symbol:`mongoc_client_write_command_with_opts`.
 * ``sessionId``: First, construct a :symbol:`mongoc_client_session_t` with :symbol:`mongoc_client_start_session`. You can begin a transaction with :symbol:`mongoc_client_session_start_transaction`, optionally with a :symbol:`mongoc_transaction_opt_t` that overrides the options inherited from |opts-source|, and use :symbol:`mongoc_client_session_append` to add the session to ``opts``. See the example code for :symbol:`mongoc_client_session_t`.
 * ``validate``: Construct a bitwise-or of all desired :symbol:`bson_validate_flags_t <bson_validate_with_error>`. Set to ``false`` to skip client-side validation of the provided BSON documents.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.
 * ``bypassDocumentValidation``: Set to ``true`` to skip server-side schema validation of the provided BSON documents.

src/libmongoc/doc/includes/replace-one-opts.txt

@@ -8,6 +8,7 @@
 * ``writeConcern``: Construct a :symbol:`mongoc_write_concern_t` and use :symbol:`mongoc_write_concern_append` to add the write concern to ``opts``. See the example code for :symbol:`mongoc_client_write_command_with_opts`.
 * ``sessionId``: First, construct a :symbol:`mongoc_client_session_t` with :symbol:`mongoc_client_start_session`. You can begin a transaction with :symbol:`mongoc_client_session_start_transaction`, optionally with a :symbol:`mongoc_transaction_opt_t` that overrides the options inherited from |opts-source|, and use :symbol:`mongoc_client_session_append` to add the session to ``opts``. See the example code for :symbol:`mongoc_client_session_t`.
 * ``validate``: Construct a bitwise-or of all desired :symbol:`bson_validate_flags_t <bson_validate_with_error>`. Set to ``false`` to skip client-side validation of the provided BSON documents.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.
 * ``bypassDocumentValidation``: Set to ``true`` to skip server-side schema validation of the provided BSON documents.
 * ``collation``: Configure textual comparisons. See :ref:`Setting Collation Order <setting_collation_order>`, and `the MongoDB Manual entry on Collation <https://docs.mongodb.com/manual/reference/collation/>`_. Collation requires MongoDB 3.2 or later, otherwise an error is returned.
 * ``hint``: A document or string that specifies the index to use to support the query predicate.

src/libmongoc/doc/includes/update-many-opts.txt

@@ -8,6 +8,7 @@
 * ``writeConcern``: Construct a :symbol:`mongoc_write_concern_t` and use :symbol:`mongoc_write_concern_append` to add the write concern to ``opts``. See the example code for :symbol:`mongoc_client_write_command_with_opts`.
 * ``sessionId``: First, construct a :symbol:`mongoc_client_session_t` with :symbol:`mongoc_client_start_session`. You can begin a transaction with :symbol:`mongoc_client_session_start_transaction`, optionally with a :symbol:`mongoc_transaction_opt_t` that overrides the options inherited from |opts-source|, and use :symbol:`mongoc_client_session_append` to add the session to ``opts``. See the example code for :symbol:`mongoc_client_session_t`.
 * ``validate``: Construct a bitwise-or of all desired :symbol:`bson_validate_flags_t <bson_validate_with_error>`. Set to ``false`` to skip client-side validation of the provided BSON documents.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.
 * ``bypassDocumentValidation``: Set to ``true`` to skip server-side schema validation of the provided BSON documents.
 * ``collation``: Configure textual comparisons. See :ref:`Setting Collation Order <setting_collation_order>`, and `the MongoDB Manual entry on Collation <https://docs.mongodb.com/manual/reference/collation/>`_. Collation requires MongoDB 3.2 or later, otherwise an error is returned.
 * ``hint``: A document or string that specifies the index to use to support the query predicate.

src/libmongoc/doc/includes/update-one-opts.txt

@@ -8,6 +8,7 @@
 * ``writeConcern``: Construct a :symbol:`mongoc_write_concern_t` and use :symbol:`mongoc_write_concern_append` to add the write concern to ``opts``. See the example code for :symbol:`mongoc_client_write_command_with_opts`.
 * ``sessionId``: First, construct a :symbol:`mongoc_client_session_t` with :symbol:`mongoc_client_start_session`. You can begin a transaction with :symbol:`mongoc_client_session_start_transaction`, optionally with a :symbol:`mongoc_transaction_opt_t` that overrides the options inherited from |opts-source|, and use :symbol:`mongoc_client_session_append` to add the session to ``opts``. See the example code for :symbol:`mongoc_client_session_t`.
 * ``validate``: Construct a bitwise-or of all desired :symbol:`bson_validate_flags_t <bson_validate_with_error>`. Set to ``false`` to skip client-side validation of the provided BSON documents.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.
 * ``bypassDocumentValidation``: Set to ``true`` to skip server-side schema validation of the provided BSON documents.
 * ``collation``: Configure textual comparisons. See :ref:`Setting Collation Order <setting_collation_order>`, and `the MongoDB Manual entry on Collation <https://docs.mongodb.com/manual/reference/collation/>`_. Collation requires MongoDB 3.2 or later, otherwise an error is returned.
 * ``hint``: A document or string that specifies the index to use to support the query predicate.

src/libmongoc/doc/mongoc_bulk_operation_set_comment.rst

@@ -0,0 +1,26 @@
+:man_page: mongoc_bulk_operation_set_comment
+
+mongoc_bulk_operation_set_comment()
+===================================
+
+Synopsis
+--------
+
+.. code-block:: c
+
+  void
+  mongoc_bulk_operation_set_comment (
+     mongoc_bulk_operation_t *bulk, const bson_value_t *comment);
+
+Parameters
+----------
+
+* ``bulk``: A :symbol:`mongoc_bulk_operation_t`.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to associate with this bulk write.
+
+Description
+-----------
+
+Assigns a comment to attach to all commands executed as part of this :doc:`bulk <mongoc_bulk_operation_t>`. The comment will appear in log messages, profiler output, and currentOp output. Comments for write commands are only supported by MongoDB 4.4+.
+
+It is prohibited to call this function after adding operations to the :doc:`bulk <mongoc_bulk_operation_t>`.

src/libmongoc/doc/mongoc_bulk_operation_t.rst

@@ -49,6 +49,7 @@ After adding all of the write operations to the ``mongoc_bulk_operation_t``, cal
     mongoc_bulk_operation_replace_one_with_opts
     mongoc_bulk_operation_set_bypass_document_validation
     mongoc_bulk_operation_set_client_session
+    mongoc_bulk_operation_set_comment
     mongoc_bulk_operation_set_hint
     mongoc_bulk_operation_set_let
     mongoc_bulk_operation_update

src/libmongoc/doc/mongoc_client_find_databases_with_opts.rst

@@ -14,6 +14,8 @@ Synopsis
 
 Fetches a cursor containing documents, each corresponding to a database on this MongoDB server.
 
+.. include:: includes/retryable-read.txt
+
 Parameters
 ----------
 
@@ -24,7 +26,7 @@ Parameters
 
 .. include:: includes/generic-opts.txt
 
-.. include:: includes/retryable-read.txt
+For a list of all options, see `the MongoDB Manual entry on the listDatabases command <https://www.mongodb.com/docs/manual/reference/command/listDatabases/>`_.
 
 Errors
 ------

src/libmongoc/doc/mongoc_client_get_database_names_with_opts.rst

@@ -16,6 +16,8 @@ Synopsis
 
 This function queries the MongoDB server for a list of known databases.
 
+.. include:: includes/retryable-read.txt
+
 Parameters
 ----------
 
@@ -27,7 +29,7 @@ Parameters
 
 .. include:: includes/generic-opts.txt
 
-.. include:: includes/retryable-read.txt
+For a list of all options, see `the MongoDB Manual entry on the listDatabases command <https://www.mongodb.com/docs/manual/reference/command/listDatabases/>`_.
 
 Errors
 ------

src/libmongoc/doc/mongoc_collection_count_documents.rst

@@ -31,6 +31,7 @@ Parameters
 .. include:: includes/read-opts.txt
 * ``skip``: An int specifying how many documents matching the ``query`` should be skipped before counting.
 * ``limit``: An int specifying the maximum number of documents to count.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Specifying a non-string value requires MongoDB 4.4 or later.
 
 Description
 -----------

src/libmongoc/doc/mongoc_collection_estimated_document_count.rst

@@ -29,6 +29,9 @@ Parameters
 .. include:: includes/read-opts.txt
 * ``skip``: An int specifying how many documents matching the ``query`` should be skipped before counting.
 * ``limit``: An int specifying the maximum number of documents to count.
+* ``comment``: A :symbol:`bson_value_t` specifying the comment to attach to this command. The comment will appear in log messages, profiler output, and currentOp output. Requires MongoDB 4.4 or later.
+
+For a list of all options, see `the MongoDB Manual entry on the count command <https://www.mongodb.com/docs/manual/reference/command/count/>`_.
 
 Description
 -----------