Skip to content

Commit e6c0007

Browse files
biniona-mongodbschmalliso
authored andcommitted
(DOCSP-15799) Write Model Strategies (#154)
1 parent 44cb00e commit e6c0007

File tree

5 files changed

+493
-10
lines changed

5 files changed

+493
-10
lines changed

source/sink-connector/configuration-properties.txt

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -50,11 +50,14 @@ See the following categories for a list of related configuration properties:
5050
* - :doc:`Sink Connector Error Handling Properties </sink-connector/configuration-properties/error-handling>`
5151
- Specify how to respond to errors and configure the dead letter queue.
5252

53+
* - :doc:`Connector Post-processor Properties </sink-connector/configuration-properties/post-processors>`
54+
- Specify transformations of Kafka topic data.
55+
5356
* - :doc:`Connector Id Strategy Properties </sink-connector/configuration-properties/id-strategy>`
5457
- Specify how the connector generates document ids.
5558

56-
* - :doc:`Connector Post-processor Properties </sink-connector/configuration-properties/post-processors>`
57-
- Specify transformations of Kafka topic data.
59+
* - :doc:`Connector Write Model Strategy Properties </sink-connector/configuration-properties/post-processors>`
60+
- Specify how the connector writes data to MongoDB.
5861

5962
* - :doc:`Topic Override Properties </sink-connector/configuration-properties/topic-override>`
6063
- Override how the connector processes data on specific Kafka topics.
@@ -77,8 +80,9 @@ for more information on these settings.
7780
Connector Topic </sink-connector/configuration-properties/kafka-topic>
7881
Connector Message Processing </sink-connector/configuration-properties/connector-message>
7982
Connector Error Handling </sink-connector/configuration-properties/error-handling>
80-
Id Strategy </sink-connector/configuration-properties/id-strategy>
8183
Post-processors <sink-connector/configuration-properties/post-processors>
84+
Id Strategy </sink-connector/configuration-properties/id-strategy>
85+
Write Model Strategy <sink-connector/configuration-properties/write-strategies>
8286
Topic Override <sink-connector/configuration-properties/topic-override>
8387
Change Data Capture <sink-connector/configuration-properties/cdc>
8488
Time Series <sink-connector/configuration-properties/time-series>

source/sink-connector/configuration-properties/post-processors.txt

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -121,7 +121,7 @@ Settings
121121
- | **Type:** string
122122
|
123123
| **Description:**
124-
| The class that specifies the ``WriteModel`` the connector should
124+
| The class that specifies the ``WriteModelStrategy`` the connector should
125125
use for :manual:`Bulk Writes </core/bulk-write-operations/index.html>`.
126126

127127
.. seealso::
@@ -137,4 +137,3 @@ Settings
137137
com.mongodb.kafka.connect.sink.writemodel.strategy.DefaultWriteModelStrategy
138138

139139
| **Accepted Values**: A fully qualified Java class name
140-
Lines changed: 120 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,120 @@
1+
=====================================
2+
Sink Connector Write Model Strategies
3+
=====================================
4+
5+
.. default-domain:: mongodb
6+
7+
.. contents:: On this page
8+
:local:
9+
:backlinks: none
10+
:depth: 2
11+
:class: singlecol
12+
13+
Overview
14+
--------
15+
16+
Use the strategies in the following table to specify how the sink connector
17+
writes data into MongoDB. You can specify a write strategy with the following
18+
configuration:
19+
20+
.. code-block:: properties
21+
22+
writemodel.strategy=<a writemodel strategy>
23+
24+
.. include:: /includes/sink-config-link.rst
25+
26+
.. _sink-connector-write-model-strategies:
27+
28+
Strategies
29+
----------
30+
31+
.. list-table::
32+
:header-rows: 1
33+
:widths: 25 75
34+
35+
* - Name
36+
- Description
37+
38+
* - | **DefaultWriteModelStrategy**
39+
|
40+
- | **Description:**
41+
| This strategy uses the ``ReplaceOneDefaultStrategy`` by default, and the ``InsertOneDefaultStrategy`` if you set the ``timeseries.timefield`` option.
42+
|
43+
| This is the default value for the ``writemodel.strategy`` configuration setting.
44+
45+
* - | **InsertOneDefaultStrategy**
46+
|
47+
- | **Description:**
48+
| Insert each sink record into MongoDB as a document.
49+
| Apply the following configuration to your sink connector to specify this setting:
50+
51+
.. code-block:: properties
52+
53+
writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.InsertOneDefaultStrategy
54+
55+
* - | **ReplaceOneDefaultStrategy**
56+
|
57+
- | **Description:**
58+
| Replaces at most one document in MongoDB that matches a sink record by the ``_id`` field. If no documents match, insert the sink record as a new document.
59+
| Apply the following configuration to your sink connector to specify this setting:
60+
61+
.. code-block:: properties
62+
63+
writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.ReplaceOneDefaultStrategy
64+
65+
* - | **ReplaceOneBusinessKeyStrategy**
66+
|
67+
- | **Description:**
68+
| Replaces at most one document that matches a sink record by a specified business key. If no documents match, insert the sink record as a new document.
69+
| Apply the following configuration to your sink connector to specify this setting:
70+
71+
.. code-block:: properties
72+
73+
writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.ReplaceOneBusinessKeyStrategy
74+
75+
| To see an example showing how to use this strategy, see our :ref:`guide on write model strategies <kafka-sink-write-model-replace-example>`.
76+
77+
* - | **DeleteOneDefaultStrategy**
78+
|
79+
- | **Description:**
80+
| Deletes at most one document that matches your sink connector's key structure by the ``_id`` field only when the document contains a null value structure.
81+
| This is implicitly specified when you set ``mongodb.delete.on.null.values=true``.
82+
| You can set this explicitly with the following configuration:
83+
84+
.. code-block:: properties
85+
86+
writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.DeleteOneDefaultStrategy
87+
88+
* - | **DeleteOneBusinessKeyStrategy**
89+
|
90+
- | **Description:**
91+
| Deletes at most one MongoDB document that matches a sink record by a business key.
92+
| Apply the following configuration to your sink connector to specify this setting:
93+
94+
.. code-block:: properties
95+
96+
writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.DeleteOneBusinessKeyStrategy
97+
98+
| To see an example showing how to use this strategy, see our :ref:`guide on write model strategies <kafka-sink-write-model-delete-example>`.
99+
100+
* - | **UpdateOneTimestampsStrategy**
101+
|
102+
- | **Description:**
103+
| Add ``_insertedTS`` (inserted timestamp) and ``_modifiedTS`` (modified timestamp) fields into documents.
104+
| Apply the following configuration to your sink connector to specify this setting:
105+
106+
.. code-block:: properties
107+
108+
writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.UpdateOneTimestampsStrategy
109+
110+
| To see an example showing how to use this strategy, see our :ref:`guide on write model strategies <kafka-sink-write-model-time-example>`.
111+
112+
* - | **UpdateOneBusinessKeyTimestampStrategy**
113+
|
114+
- | **Description:**
115+
| Add ``_insertedTS`` (inserted timestamp) and ``_modifiedTS`` (modified timestamp) fields into documents that match a business key.
116+
| Apply the following configuration to your sink connector to specify this setting:
117+
118+
.. code-block:: properties
119+
120+
writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.UpdateOneBusinessKeyTimestampStrategy

source/sink-connector/fundamentals.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@ Fundamentals
66
:titlesonly:
77
:maxdepth: 1
88

9-
Write Strategies </sink-connector/fundamentals/write-strategies>
9+
Write Model Strategies </sink-connector/fundamentals/write-strategies>
1010
Post-processors </sink-connector/fundamentals/post-processors>
1111
Data Transformations </sink-connector/fundamentals/data-transformations>
1212
Error Handling Strategies </sink-connector/fundamentals/error-handling-strategies>

0 commit comments

Comments
 (0)