Skip to content

Commit 0717d18

Browse files
committed
Snooty setup (#67)
* DOP-1608: set up kafka v1.3 for snooty build * test
1 parent a5375f5 commit 0717d18

8 files changed

+123
-115
lines changed

snooty.toml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
name = "kafka-connector"
2+
title = "MongoDB Kafka Connector"
3+
intersphinx = ["https://docs.mongodb.com/manual/objects.inv"]
4+
5+
toc_landing_pages = ["/kafka-sink"]

source/includes/externalize-secrets.rst

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,4 @@
1-
.. admonition:: Avoid Exposing Your Authentication Credentials
2-
:class: important
1+
.. important:: Avoid Exposing Your Authentication Credentials
32

43
To avoid exposing your authentication credentials in your
54
``connection.uri`` setting, use a

source/kafka-docker-example.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,7 @@ http://localhost:9021/ and navigate to the cluster topics.
185185
Next, explore the collection data in the MongoDB replica set:
186186

187187
* In your local shell, navigate to the ``docker`` directory from which you
188-
ran the ``docker-compose`` commands and connect to the `mongo1` MongoDB
188+
ran the ``docker-compose`` commands and connect to the ``mongo1`` MongoDB
189189
instance using the following command:
190190

191191
.. code-block:: none

source/kafka-sink-data-formats.txt

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -103,8 +103,7 @@ without schema** data format. The Kafka topic data must be in JSON format.
103103
value.converter=org.apache.kafka.connect.json.JsonConverter
104104
value.converter.schemas.enable=false
105105

106-
.. admonition:: Choose the appropriate data format
107-
:class: note
106+
.. note:: Choose the appropriate data format
108107

109108
When you specify **JSON without Schema**, any JSON schema objects such
110109
as ``schema`` or ``payload`` are read explicitly rather than as a

source/kafka-sink-postprocessors.txt

Lines changed: 24 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -40,32 +40,37 @@ class or use one of the following pre-built ones:
4040
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.DocumentIdAdder``
4141
| Uses a configured *strategy* to insert an ``_id`` field.
4242

43-
.. seealso:: :ref:`Strategy options and configuration <config-document-id-adder>`.
43+
.. seealso::
4444

45+
:ref:`Strategy options and configuration <config-document-id-adder>`.
4546
* - BlockListKeyProjector
4647
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.BlockListKeyProjector``
4748
| Removes matching key fields from the sink record.
4849

49-
.. seealso:: :ref:`Configuration <config-blocklist-allowlist>` and :ref:`Example <blocklist-example>`.
50+
.. seealso::
5051

52+
:ref:`Configuration <config-blocklist-allowlist>` and :ref:`Example <blocklist-example>`.
5153
* - BlockListValueProjector
5254
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.BlockListValueProjector``
5355
| Removes matching value fields from the sink record.
5456

55-
.. seealso:: :ref:`Configuration <config-blocklist-allowlist>` and :ref:`Example <blocklist-example>`.
57+
.. seealso::
5658

59+
:ref:`Configuration <config-blocklist-allowlist>` and :ref:`Example <blocklist-example>`.
5760
* - AllowListKeyProjector
5861
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.AllowListKeyProjector``
5962
| Includes only matching key fields from the sink record.
6063

61-
.. seealso:: :ref:`Configuration <config-blocklist-allowlist>` and :ref:`Example <allowlist-example>`.
64+
.. seealso::
6265

66+
:ref:`Configuration <config-blocklist-allowlist>` and :ref:`Example <allowlist-example>`.
6367
* - AllowListValueProjector
6468
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.AllowListValueProjector``
6569
| matching value fields from the sink record.
6670

67-
.. seealso:: :ref:`Configuration <config-blocklist-allowlist>` and :ref:`Example <allowlist-example>`.
71+
.. seealso::
6872

73+
:ref:`Configuration <config-blocklist-allowlist>` and :ref:`Example <allowlist-example>`.
6974
* - KafkaMetaAdder
7075
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.KafkaMetaAdder``
7176
| Adds a field composed of the concatenation of Kafka topic, partition, and offset to the document.
@@ -74,14 +79,16 @@ class or use one of the following pre-built ones:
7479
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.field.renaming.RenameByMapping``
7580
| Renames fields that are an exact match to a specified field name in the key or value document.
7681

77-
.. seealso:: :ref:`Renaming configuration <config-field-renaming>` and :ref:`Example <field-renaming-mapping-example>`.
82+
.. seealso::
7883

84+
:ref:`Renaming configuration <config-field-renaming>` and :ref:`Example <field-renaming-mapping-example>`.
7985
* - RenameByRegex
8086
- | Full Path: ``com.mongodb.kafka.connect.sink.processor.field.renaming.RenameByRegex``
8187
| Renames fields that match a regular expression.
8288

83-
.. seealso:: :ref:`Renaming configuration <config-field-renaming>` and :ref:`Example <field-renaming-regex-example>`.
89+
.. seealso::
8490

91+
:ref:`Renaming configuration <config-field-renaming>` and :ref:`Example <field-renaming-regex-example>`.
8592

8693
You can configure the post processor chain by specifying an ordered,
8794
comma separated list of fully-qualified ``PostProcessor`` class names:
@@ -178,8 +185,7 @@ To define a custom strategy, create a class that implements the interface
178185
and provide the fully-qualified path to the ``document.id.strategy``
179186
setting.
180187

181-
.. admonition:: Selected strategy may have implications on delivery semantics
182-
:class: note
188+
.. note:: Selected strategy may have implications on delivery semantics
183189

184190
BSON ObjectId or UUID strategies can only guarantee at-least-once
185191
delivery since new ids would be generated on retries or re-processing.
@@ -324,10 +330,10 @@ The previous example projection configurations demonstrated exact
324330
string matching on field names. The projection ``list`` setting also supports
325331
the following wildcard patterns matching on field names:
326332

327-
* "``*``" (`star`): matches a string of any length for the level in the
333+
* "``*``" (``star``): matches a string of any length for the level in the
328334
document in which it is specified.
329335

330-
* "``**``" (`double star`): matches the current and all nested levels from
336+
* "``**``" (``double star``): matches the current and all nested levels from
331337
which it is specified.
332338

333339
The examples below demonstrate how to use each wildcard pattern and the
@@ -637,7 +643,7 @@ The post processor applied the following changes:
637643
subdocuments of ``crepes`` are matched. In the matched fields, all
638644
instances of "purchased" are replaced with "quantity".
639645

640-
.. admonition:: Ensure renaming does not result in duplicate keys in the same document
646+
.. tip:: Ensure renaming does not result in duplicate keys in the same document
641647

642648
The renaming post processors update the key fields of a JSON document
643649
which can result in duplicate keys within a document. They skip the
@@ -650,9 +656,9 @@ Custom Write Models
650656

651657
A **write model** defines the behavior of bulk write operations made on a
652658
MongoDB collection. The default write model for the connector is
653-
:java-docs-latest:`ReplaceOneModel
659+
:java-docs:`ReplaceOneModel
654660
<javadoc/com/mongodb/client/model/ReplaceOneModel.html>` with
655-
:java-docs-latest:`ReplaceOptions </javadoc/com/mongodb/client/model/ReplaceOptions.html>`
661+
:java-docs:`ReplaceOptions </javadoc/com/mongodb/client/model/ReplaceOptions.html>`
656662
set to upsert mode.
657663

658664
You can override the default write model by specifying a custom one in the
@@ -674,8 +680,9 @@ strategies are provided with the connector:
674680
- | Replaces at most one document that matches filters provided by the ``document.id.strategy`` setting.
675681
| Set the following configuration: ``writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.ReplaceOneBusinessKeyStrategy``
676682

677-
.. seealso:: Example of usage in :ref:`writemodel-strategy-business-key`.
683+
.. seealso::
678684

685+
Example of usage in :ref:`writemodel-strategy-business-key`.
679686
* - DeleteOneDefaultStrategy
680687
- | Deletes at most one document that matches the id specified by the ``document.id.strategy`` setting, only when the document contains a null value record.
681688
| Implicitly specified when the configuration setting ``mongodb.delete.on.null.values=true`` is set.
@@ -685,8 +692,9 @@ strategies are provided with the connector:
685692
- | Add ``_insertedTS`` (inserted timestamp) and ``_modifiedTS`` (modified timestamp) fields into documents.
686693
| Set the following configuration: ``writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.UpdateOneTimestampsStrategy``
687694

688-
.. seealso:: Example of usage in :ref:`writemodel-strategy-timestamps`.
695+
.. seealso::
689696

697+
Example of usage in :ref:`writemodel-strategy-timestamps`.
690698
* - UpdateOneBusinessKeyTimestampStrategy
691699
- | Add ``_insertedTS`` (inserted timestamp) and ``_modifiedTS`` (modified timestamp) fields into documents that match the filters provided by the ``document.id.strategy`` setting.
692700
| Set the following configuration: ``writemodel.strategy=com.mongodb.kafka.connect.sink.writemodel.strategy.UpdateOneBusinessKeyTimestampStrategy``

source/kafka-sink-properties.txt

Lines changed: 12 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -43,15 +43,13 @@ data to sink to MongoDB. For an example configuration file, see
4343
- string
4444
- | A regular expression that matches the Kafka topics that the sink connector should watch.
4545

46-
.. example::
46+
The following regex matches topics such as
47+
"activity.landing.clicks" and "activity.support.clicks",
48+
but not "activity.landing.views" or "activity.clicks":
4749

48-
| The following regex matches topics such as
49-
| "activity.landing.clicks" and "activity.support.clicks",
50-
| but not "activity.landing.views" or "activity.clicks":
50+
.. code-block:: none
5151

52-
.. code-block:: none
53-
54-
topics.regex=activity\\.\\w+\\.clicks$
52+
topics.regex=activity\\.\\w+\\.clicks$
5553

5654
| *Required*
5755
| **Note:** You can only define either ``topics`` or ``topics.regex``.
@@ -61,11 +59,9 @@ data to sink to MongoDB. For an example configuration file, see
6159
- string
6260
- | A :manual:`MongoDB connection URI string </reference/connection-string/#standard-connection-string-format>`.
6361

64-
.. example::
65-
66-
.. code-block:: none
62+
.. code-block:: none
6763

68-
mongodb://username:password@localhost/
64+
mongodb://username:password@localhost/
6965

7066
.. include:: /includes/externalize-secrets.rst
7167

@@ -146,11 +142,9 @@ data to sink to MongoDB. For an example configuration file, see
146142
- string
147143
- | An inline JSON array with objects describing field name mappings.
148144

149-
.. example::
145+
.. code-block:: none
150146

151-
.. code-block:: none
152-
153-
[ { "oldName":"key.fieldA", "newName":"field1" }, { "oldName":"value.xyz", "newName":"abc" } ]
147+
[ { "oldName":"key.fieldA", "newName":"field1" }, { "oldName":"value.xyz", "newName":"abc" } ]
154148

155149
| **Default**: ``[]``
156150
| **Accepted Values**: A valid JSON array
@@ -159,11 +153,9 @@ data to sink to MongoDB. For an example configuration file, see
159153
- string
160154
- | An inline JSON array containing regular expression statement objects.
161155

162-
.. example::
163-
164-
.. code-block:: none
156+
.. code-block:: none
165157

166-
[ {"regexp":"^key\\\\..*my.*$", "pattern":"my", "replace":""}, {"regexp":"^value\\\\..*$", "pattern":"\\\\.", "replace":"_"} ]
158+
[ {"regexp":"^key\\\\..*my.*$", "pattern":"my", "replace":""}, {"regexp":"^value\\\\..*$", "pattern":"\\\\.", "replace":"_"} ]
167159

168160
| **Default**: ``[]``
169161
| **Accepted Values**: A valid JSON array
@@ -242,8 +234,7 @@ data to sink to MongoDB. For an example configuration file, see
242234
- int
243235
- | The maximum number of tasks that should be created for this connector. The connector may create fewer tasks if it cannot handle the specified level of parallelism.
244236

245-
.. admonition:: Messages May Be Processed Out of Order For Values Greater Than 1
246-
:class: important
237+
.. important:: Messages May Be Processed Out of Order For Values Greater Than 1
247238

248239
If you specify a value greater than ``1``, the connector
249240
enables parallel processing of the tasks. If your topic has

0 commit comments

Comments
 (0)