Skip to content

DOCSP-41769: Improved bulk write API #590

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/vale-tdbx.yml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,9 @@ jobs:
- name: checkout
uses: actions/checkout@master

- name: Install docutils
run: sudo apt-get install -y docutils

- id: files
uses: masesgroup/retrieve-changed-files@v2
with:
Expand Down
214 changes: 195 additions & 19 deletions source/fundamentals/crud/write-operations/bulk.txt
Original file line number Diff line number Diff line change
Expand Up @@ -16,22 +16,33 @@
In this guide, you can learn how to use bulk operations in the
MongoDB Java Driver.

To perform a create, replace, update, or delete operation,
use its corresponding method. For example, to insert one document,
update multiple documents, and delete one document in your collection,
use the ``insertOne()``, ``updateMany()`` and ``deleteOne()`` methods.
To perform a single create, replace, update, or delete operation, you can use
the corresponding method. For example, to insert one document and replace one
document, you can use the ``insertOne()`` and ``replaceOne()`` methods. The
``MongoClient`` makes a call to the database for each operation. You can reduce
the number of calls to one by using bulk write operations.

The ``MongoClient`` performs these operations by making a call for each
operation to the database. You can reduce the number of calls to the
database to one by using bulk operations.
You can perform bulk write operations by using the bulk write API in the
{+driver-short+} driver to perform multiple data changes in one call. You can
perform bulk operations at the following levels:

Performing Bulk Operations
--------------------------
- :ref:`Collection Level <java-sync-coll-bulk-write>`: You can use the
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- :ref:`Collection Level <java-sync-coll-bulk-write>`: You can use the
- :ref:`Collection <java-sync-coll-bulk-write>`: You can use the

``MongoCollection.bulkWrite()`` API to perform bulk write operations on a
collection.

- :ref:`Client Level <java-sync-client-bulk-write>`: In {+mdb-server+} version
8.0 or later, you can use the ``MongoClient.bulkWrite()`` API to perform bulk write
operations on multiple collections and databases in the same cluster.

.. _java-sync-coll-bulk-write:

Collection Bulk Write
---------------------

Bulk operations consist of a large number of write operations. To perform
a bulk operation, pass a ``List`` of ``WriteModel`` documents to the
``bulkWrite()`` method. A ``WriteModel`` is a model that represents any
of the write operations.
a bulk operation at the collection level, pass a ``List`` of ``WriteModel``
documents to the ``MongoCollection.bulkWrite()`` method. A ``WriteModel`` is a model that
represents any of the write operations.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
represents any of the write operations.
represents a write operation.


The following sections show how to create and use each ``WriteModel``
document. The examples in each section use the following documents in the
Expand All @@ -44,7 +55,7 @@
{ "_id": 8, "name": "Shayla Ray", "age": 20 }

For more information about the methods and classes mentioned in this section,
see the following API Documentation:
see the following API documentation:

- `bulkWrite() <{+api+}/apidocs/mongodb-driver-sync/com/mongodb/client/MongoCollection.html#bulkWrite(java.util.List,com.mongodb.client.model.BulkWriteOptions)>`__
- `WriteModel <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/WriteModel.html>`__
Expand Down Expand Up @@ -100,7 +111,7 @@
For more information about the methods and classes mentioned in this section,
see the `InsertOneModel
<{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/InsertOneModel.html>`__
API Documentation.
API documentation.

Replace Operation
~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -132,7 +143,7 @@
For more information about the methods and classes mentioned in this section,
see the following resources:

- `ReplaceOneModel <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/ReplaceOneModel.html>`__ API Documentation
- `ReplaceOneModel <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/ReplaceOneModel.html>`__ API documentation
- :manual:`Unique indexes </core/index-unique/>` Server Manual Explanation

Update Operation
Expand Down Expand Up @@ -168,8 +179,8 @@
For more information about the methods and classes mentioned in this section,
see the following resources:

- `UpdateOneModel <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/UpdateOneModel.html>`__ API Documentation
- `UpdateManyModel <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/UpdateManyModel.html>`__ API Documentation
- `UpdateOneModel <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/UpdateOneModel.html>`__ API documentation
- `UpdateManyModel <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/UpdateManyModel.html>`__ API documentation
- :manual:`unique indexes </core/index-unique/>` Server Manual Explanation

Delete Operation
Expand Down Expand Up @@ -202,7 +213,7 @@
:end-before: end deleteDocumentsExample

For more information about the methods and classes mentioned in this section,
see the following API Documentation:
see the following API documentation:

- `DeleteOneModel <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/DeleteOneModel.html>`__
- `DeleteManyModel <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/DeleteManyModel.html>`__
Expand Down Expand Up @@ -288,14 +299,159 @@
{ "_id": 6, "name": "Zaynab Omar", "age": 37 }

For more information about the methods and classes mentioned in this section,
see the following API Documentation:
see the following API documentation:

- `BulkWriteOptions <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/BulkWriteOptions.html>`__
- `ordered() <{+api+}/apidocs/mongodb-driver-core/com/mongodb/client/model/BulkWriteOptions.html#ordered(boolean)>`__

.. _java-sync-client-bulk-write:

Client Bulk Write
-----------------

When connecting to a deployment running {+mdb-server+} 8.0 or later,
you can use the ``MongoClient.bulkWrite()`` method to write
to multiple databases and collections in the same cluster.

In the examples in preceding sections of this page, the ``MongoCollection.bulkWrite()`` method
takes a list of ``WriteModel`` instances in which the specified subclass of
``WriteModel`` represents the corresponding write operation. For example, an
instance of ``InsertOneModel`` represents an operation to insert one document.

Similarly, the ``MongoClient.bulkWrite()`` method takes a
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S: unfortunately, i think it's worth repeating the info from earlier in the page just to users don't have to read earlier sections if they don't need to. if you have a lot of repeated content, you can always put it in an includes directive 😎

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

honestly, I'm not sure how much this information is relevant so I will remove it 👍

list of ``ClientNamespacedWriteModel`` instances to represent different write operations.
However, you do not have to specify the subclass that represents the corresponding

Check failure on line 323 in source/fundamentals/crud/write-operations/bulk.txt

View workflow job for this annotation

GitHub Actions / TDBX Vale rules

[vale] reported by reviewdog 🐶 [MongoDB.ConciseTerms] 'must' is preferred over 'have to'. Raw Output: {"message": "[MongoDB.ConciseTerms] 'must' is preferred over 'have to'.", "location": {"path": "source/fundamentals/crud/write-operations/bulk.txt", "range": {"start": {"line": 323, "column": 21}}}, "severity": "ERROR"}
write operation. Instead, the ``ClientNamespacedWriteModel`` object contains different
methods to represent different write operations, such as ``ClientNamespacedWriteModel.insertOne()`` or
``ClientNamespacedWriteModel.updateOne()``.

These methods take a ``MongoNamespace`` object that defines which
database and collection to write to, and a ``Document`` object that defines
information about the write operation. Some methods, such as ``updateOne()`` and
``replaceOne()``, also take a ``Filters`` object that defines the subset of
documents to be updated or replaced.

The following sections describe how to use the client ``bulkWrite()`` method.

Insert Example
~~~~~~~~~~~~~~

This example shows how to use the ``bulkWrite()`` method to insert
two documents. One document is inserted into the ``db.people`` collection, while
the other document is inserted into the ``db.things`` collection.
The ``MongoNamespace`` instance defines the databases and collections that
each write operation applies to.

.. code-block:: java

MongoNamespace peopleNamespace = new MongoNamespace("db", "people");
MongoNamespace thingsNamespace = new MongoNamespace("db", "things");

List<ClientNamespacedWriteModel> bulkOperations = new ArrayList<>();

bulkOperations.add(
ClientNamespacedWriteModel.insertOne(
peopleNamespace,
new Document("name", "Julia Smith")
)
);

bulkOperations.add(
ClientNamespacedWriteModel.insertOne(
thingsNamespace,
new Document("object", "washing machine")
)
);

ClientBulkWriteResult result = mongoClient.bulkWrite(bulkOperations);

.. TODO: link documentation

Replace Example
~~~~~~~~~~~~~~~

The following example shows how to use the ``bulkWrite()`` method to replace
existing documents in the ``db.people`` and ``db.things`` collections.

.. code-block:: java

MongoNamespace peopleNamespace = new MongoNamespace("db", "people");
MongoNamespace thingsNamespace = new MongoNamespace("db", "things");

List<ClientNamespacedWriteModel> bulkOperations = new ArrayList<>();

bulkOperations.add(ClientNamespacedWriteModel.replaceOne(
peopleNamespace,
Filters.eq("_id", 1),
new Document("name", "Frederic Hilbert")
)
);

bulkOperations.add(ClientNamespacedWriteModel.replaceOne(
thingsNamespace,
Filters.eq("_id", 1),
new Document("object", "potato")
)
);

ClientBulkWriteResult result = mongoClient.bulkWrite(bulkOperations);

After this example runs successfully, the document that has an ``_id`` value of ``1``
in the ``people`` collection is replaced with a new document. The document in
the ``things`` collection that has an ``_id`` value of ``1``
is replaced with a new document.

.. _java-sync-client-bulk-write-options:

Bulk Write Options
~~~~~~~~~~~~~~~~~~

You can pass an instance of ``ClientBulkWriteOptions`` to the ``bulkWrite()``
method to specify options when running the bulk write operations.

Order of Execution Example
``````````````````````````

By default, the driver performs write operations in the order that you specify them until
an error occurs, or until they execute successfully. However, you can pass
``false`` to the ``ordered()`` method on the ``ClientBulkWriteOptions``
interface to perform write operations in an unordered way. When using the unordered
option, an error-producing operation does not block execution of other bulk
write operations.

The following code sets the ``ordered()`` method on an
instance of ``ClientBulkWriteOptions`` and performs a bulk write operation to
insert multiple documents.

.. code-block:: java

MongoNamespace namespace = new MongoNamespace("db", "people");

ClientBulkWriteOptions options = new ClientBulkWriteOptions().ordered(false);

List<ClientNamespacedWriteModel> bulkOperations = new ArrayList<>();

bulkOperations.add(ClientNamespacedWriteModel.insertOne(namespace, new Document("_id", 1).append("name", "Rudra Suraj)));

// Causes a duplicate key error
bulkOperations.add(ClientNamespacedWriteModel.insertOne(namespace, new Document("_id", 1).append("name", "Mario Bianchi")));

bulkOperations.add(ClientNamespacedWriteModel.insertOne(namespace, new Document("name", "Wendy Zhang")));

ClientBulkWriteResult result = mongoClient.bulkWrite(bulkOperations, options);

Even though the write operation inserting a document with a duplicate key results
in an error, the other operations are executed because the write operation is
unordered.

.. TODO: link documentation

Summary
-------

``MongoCollection.bulkWrite()``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

To perform a bulk operation, you create and pass a list of
``WriteModel`` documents to the ``bulkWrite()`` method.

Expand All @@ -308,3 +464,23 @@
- Ordered, which performs the bulk operations in order until an error occurs, if any
- Unordered, which performs all the bulk operations in any order and reports errors
at the end, if any

``MongoClient.bulkWrite()``
~~~~~~~~~~~~~~~~~~~~~~~~~~~

When connecting to a deployment running {+mdb-server+} version 8.0 or later, you
can use the ``MongoClient.bulkWrite()`` method to perform bulk operations on multiple
databases and collections at once.

This method uses the ``ClientNamespacedWriteModel`` and its methods
``insertOne()``, ``updateOne()``, ``updateMany()``, ``replaceOne()``,
``deleteOne()``, and ``deleteMany()``.

The method can also take a ``ClientBulkWriteOptions`` object to specify different
options for how the command is executed.

To learn more about the client ``bulkWrite`` command, see the
:manual:`bulkWrite() <reference/command/bulkWrite/>` method reference in the {+mdb-server+}
Manual.

.. TODO: Add API Documentation
13 changes: 13 additions & 0 deletions source/whats-new.txt
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ What's New

Learn what's new in:

* :ref:`Version 5.3 <java-version-5.3>`
* :ref:`Version 5.2.1 <java-version-5.2.1>`
* :ref:`Version 5.2 <java-version-5.2>`
* :ref:`Version 5.1.3 <java-version-5.1.3>`
Expand All @@ -29,6 +30,18 @@ Learn what's new in:
* :ref:`Version 4.11 <version-4.11>`
* :ref:`Version 4.10 <version-4.10>`

.. _java-version-5.3:

What's New in 5.3
-----------------

The 5.3 {+driver-short+} release introduces the following improvements, features,
and fixes:

- Client bulk write API that allows you to perform write operations on multiple
databases and collections at once. To learn more about this feature, see the
:ref:`java-sync-client-bulk-write` section of the Bulk Operations guide.

.. _java-version-5.2.1:

What's New in 5.2.1
Expand Down
Loading