Skip to content

Commit 3ec1277

Browse files
Chris Choschmalliso
authored andcommitted
DOCSP-14391: dynamic collection naming settings (#85)
* DOCSP-14391: dynamic collection naming settings
1 parent b517fc3 commit 3ec1277

File tree

1 file changed

+84
-0
lines changed

1 file changed

+84
-0
lines changed

source/kafka-sink-properties.txt

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,21 @@ data to sink to MongoDB. For an example configuration file, see
8585
| *Required*
8686
| **Accepted Values**: A MongoDB collection name
8787

88+
* - namespace.mapper
89+
- string
90+
- | The class name of the class that specifies which database or
91+
collection in which to sink the data. The
92+
``DefaultNamespaceMapper`` uses the ``database`` and ``collection``
93+
settings.
94+
95+
.. seealso::
96+
97+
- :ref:`FieldPathNamespaceMapper settings <fieldpathnamespacemapper-settings>`
98+
99+
| **Default**: ``com.mongodb.kafka.connect.sink.namespace.mapping.DefaultNamespaceMapper``
100+
| **Accepted Values**: A fully qualified Java class name of a class
101+
that implements the ``NamespaceMapper`` interface.
102+
88103
* - document.id.strategy
89104
- string
90105
- | The class name of the class that generates a unique document ``_id`` field.
@@ -308,6 +323,75 @@ the following behavior for data consumed from ``topicA``:
308323
``BlockList`` projection type.
309324

310325

326+
.. _fieldpathnamespacemapper-settings:
327+
328+
FieldPathNamespaceMapper Settings
329+
---------------------------------
330+
331+
You can specify which namespace (database and/or collection) to sink
332+
a document based on its field values using the
333+
``FieldPathNamespaceMapper``. To enable this mapper, set the
334+
``namespace.mapper`` configuration setting to the class name as shown
335+
below:
336+
337+
.. code-block:: properties
338+
339+
namespace.mapper=com.mongodb.kafka.connect.sink.topic.mapping.FieldPathNamespaceMapper
340+
341+
The ``FieldPathNamespaceMapper`` requires the following:
342+
343+
- at least one of the mapping configurations to a database or collection is set
344+
- only one of the ``key`` or ``value`` mapping configurations is set for a
345+
mapping to a database
346+
- only one of the ``key`` or ``value`` mapping configurations is set for a
347+
mapping to a collection
348+
349+
You can use the following settings to customize the behavior of the
350+
``FieldPathNamespaceMapper``:
351+
352+
.. list-table::
353+
:header-rows: 1
354+
:stub-columns: 1
355+
:widths: 2 1 4
356+
357+
* - Name
358+
- Type
359+
- Description
360+
361+
* - namespace.mapper.key.database.field
362+
- string
363+
- | The name of the key document field that specifies the name of the
364+
database to write to.
365+
366+
* - namespace.mapper.key.collection.field
367+
- string
368+
- | The name of the key document field that specifies the name of the
369+
collection to write to.
370+
371+
* - namespace.mapper.value.database.field
372+
- string
373+
- | The name of the value document field that specifies the name of
374+
the database to write to.
375+
376+
* - namespace.mapper.value.collection.field
377+
- string
378+
- | The name of the value document field that specifies the name of
379+
the collection to write to.
380+
381+
* - namespace.mapper.error.if.invalid
382+
- boolean
383+
- | Whether to throw an exception if the document is missing the
384+
mapped field or if it contains an invalid BSON type.
385+
| When set to ``true``, the connector does not process the record
386+
and may halt or skip processing depending on the related settings.
387+
| When set to ``false`` and a document is missing the mapped field
388+
or if it contains an invalid BSON type, the connector defaults to
389+
writing to the specified ``database`` and ``collection`` settings.
390+
|
391+
| **Default**: ``false``
392+
| **Accepted Values**: ``true`` or ``false``
393+
394+
311395
Dead Letter Queue Configuration Settings
312396
----------------------------------------
313397

0 commit comments

Comments
 (0)