Skip to content

Commit 8cdaa48

Browse files
authored
DOCSP-20912: create material on bulk writes (#74)
* DOCSP-20912: create material on bulk writes * MW PR fixes 1 * fix embiguous linked target * MW PR fixes 2 * typo fix
1 parent b1f7773 commit 8cdaa48

File tree

3 files changed

+44
-36
lines changed

3 files changed

+44
-36
lines changed

source/sink-connector/configuration-properties/connector-message.txt

Lines changed: 2 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -77,25 +77,8 @@ Settings
7777
When set to ``true``, the default value, the
7878
connector writes a batch of records as an ordered bulk write
7979
operation.
80-
| To learn more about ordered and unordered
81-
bulk write operations, see
82-
`Bulk Write Operations <{+connector_driver_url_base+}fundamentals/crud/write-operations/bulk/#order-of-execution>`__
83-
in the MongoDB Java driver documentation.
84-
85-
.. tip:: Performance Benefits
86-
87-
If you can write your data to MongoDB in any order, use unordered
88-
bulk write operations. Unordered bulk writes provide the following
89-
benefits:
90-
91-
- MongoDB can execute unordered bulk writes in parallel
92-
- If you configured your connector to tolerate errors, unordered bulk writes
93-
help you :ref:`lose less data <kafka-sink-bulk-ordered-data-loss>`.
94-
95-
To learn more about the performance benefits of unordered
96-
bulk write operations, see :manual:`Unordered Operations </reference/method/Bulk/#unordered-operations>`
97-
in the MongoDB Manual.
98-
80+
| To learn more about bulk write operations, see the :ref:`Write
81+
Model Strategies page <sink-connector-bulk-write-ops>`.
9982
|
10083
| **Default**: ``true``
10184
| **Accepted Values**: ``true`` or ``false``

source/sink-connector/fundamentals/error-handling-strategies.txt

Lines changed: 3 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -88,19 +88,9 @@ option:
8888
.. warning:: Ordered Bulk Writes Can Result in Skipped Messages
8989

9090
If you set your connector to tolerate errors and use ordered bulk writes, you
91-
may lose data. When an ordered bulk write fails, the connector skips the
92-
remaining writes in that batch and resumes processing the next batch.
93-
94-
If you set your connector to tolerate errors and use unordered bulk writes,
95-
you lose less data. Your connector attempts to perform every write in the
96-
batch even if some writes fail.
97-
98-
When using unordered bulk writes, ensure you can perform the write operations
99-
within each batch in any order.
100-
101-
To configure your connector to use unordered bulk writes, set the
102-
``bulk.write.ordered`` property to ``false``. To learn more about this
103-
property, see the :ref:`<sink-configuration-message-processing>` page.
91+
may lose data. If you set your connector to tolerate errors and use unordered bulk writes,
92+
you lose less data. To learn more about bulk write operations, see
93+
the :ref:`Write Model Strategies page <sink-connector-bulk-write-ops>`.
10494

10595
.. _kafka-sink-errors-dlq:
10696

source/sink-connector/fundamentals/write-strategies.txt

Lines changed: 39 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -35,11 +35,46 @@ To learn how to modify the sink records your connector receives before your
3535
connector writes them to MongoDB, read the guide on
3636
:ref:`Sink Connector Post Processors <sink-fundamentals-post-processors>`.
3737

38-
To learn more about bulk writes, see the MongoDB Java driver guide on
39-
`Bulk Writes <{+connector_driver_url_base+}fundamentals/crud/write-operations/bulk/>`__.
40-
4138
To see a write model strategy implementation, see the source code of the
42-
:github:`InsertOneDefaultStrategy class <mongodb/mongo-kafka/blob/master/src/main/java/com/mongodb/kafka/connect/sink/writemodel/strategy/InsertOneDefaultStrategy.java>`.
39+
:github:`InsertOneDefaultStrategy class
40+
<mongodb/mongo-kafka/blob/master/src/main/java/com/mongodb/kafka/connect/sink/writemodel/strategy/InsertOneDefaultStrategy.java>`.
41+
42+
.. _sink-connector-bulk-write-ops:
43+
44+
Bulk Write Operations
45+
---------------------
46+
47+
The sink connector writes data to MongoDB using bulk write operations.
48+
Bulk writes group multiple write operations, such as inserts,
49+
updates, or deletes, together.
50+
51+
By default, the sink connector performs ordered bulk writes, which
52+
guarantee the order of data changes. In an ordered bulk write, if any
53+
write operation results in an error, the connector skips the remaining
54+
writes in that batch.
55+
56+
If you don't need to guarantee the order of data changes, you can
57+
set the ``bulk.write.ordered`` setting to ``false`` so that the
58+
connector performs unordered bulk writes. The sink connector performs
59+
unordered bulk writes in parallel, which can improve performance.
60+
61+
In addition, when you enable unordered bulk writes and set the
62+
``errors.tolerance`` setting to ``all``, even if any write
63+
operation in your bulk write fails, the connector continues to
64+
perform the remaining write operations in the batch that do not return
65+
errors.
66+
67+
.. tip::
68+
69+
To learn more about the ``bulk.write.ordered`` setting, see the
70+
:ref:`Connector Message Processing Properties
71+
<sink-configuration-message-processing>`.
72+
73+
To learn more about bulk write operations, see the following
74+
documentation:
75+
76+
- :manual:`Server manual entry on ordered and unordered bulk operations </reference/method/Bulk/#ordered-and-unordered-bulk-operations>`.
77+
- `Bulk write operations in Java <{+connector_driver_url_base+}fundamentals/crud/write-operations/bulk/#order-of-execution>`__
4378

4479
How to Specify Write Model Strategies
4580
-------------------------------------

0 commit comments

Comments
 (0)