@@ -196,9 +196,9 @@ an example source connector configuration file, see
196
196
197
197
* - output.schema.key
198
198
- string
199
- - The `Avro schema <https://avro.apache.org/docs/current/spec.html#schemas>`__
199
+ - The `Avro schema <https://avro.apache.org/docs/current/spec.html#schemas>`__
200
200
definition for the key document of the SourceRecord.
201
-
201
+
202
202
**Default**:
203
203
204
204
.. code-block:: json
@@ -213,13 +213,13 @@ an example source connector configuration file, see
213
213
214
214
* - output.schema.value
215
215
- string
216
- - The `Avro schema <https://avro.apache.org/docs/current/spec.html#schemas>`__
216
+ - The `Avro schema <https://avro.apache.org/docs/current/spec.html#schemas>`__
217
217
definition for the value document of the SourceRecord.
218
218
219
219
**Default**:
220
220
221
221
.. code-block:: json
222
-
222
+
223
223
{
224
224
"name": "ChangeStream",
225
225
"type": "record",
@@ -300,8 +300,8 @@ an example source connector configuration file, see
300
300
- string
301
301
- | Prefix to prepend to database & collection names to generate the name of the Kafka topic to publish data to.
302
302
303
- .. seealso::
304
-
303
+ .. seealso::
304
+
305
305
:ref:`Topic naming example <topic-naming-example>`.
306
306
307
307
| **Default**: ""
@@ -329,6 +329,11 @@ an example source connector configuration file, see
329
329
330
330
copy.existing.namespace.regex=stats\.page.*
331
331
332
+ Note that in the example above, the "\\" character in the example
333
+ escapes the following "." character in the regular expression.
334
+ For more information on how to build regular expressions, see the
335
+ `Java SE documentation on Patterns <https://docs.oracle.com/javase/8/docs/api/java/util/regex/Pattern.html>`__.
336
+
332
337
| **Default**: ""
333
338
| **Accepted Values**: A valid regular expression
334
339
@@ -409,9 +414,9 @@ an example source connector configuration file, see
409
414
410
415
* - heartbeat.interval.ms
411
416
- int
412
- - The length of time in milliseconds between sending heartbeat messages to
413
- record a post batch resume token when no source records have been published.
414
- This can improve the resumability of the connector for low volume namespaces.
417
+ - The length of time in milliseconds between sending heartbeat messages to
418
+ record a post batch resume token when no source records have been published.
419
+ This can improve the resumability of the connector for low volume namespaces.
415
420
Use ``0`` to disable.
416
421
417
422
| **Default**: ``0``
@@ -420,7 +425,7 @@ an example source connector configuration file, see
420
425
* - heartbeat.topic.name
421
426
- string
422
427
- The name of the topic to write heartbeat messages to.
423
-
428
+
424
429
| **Default**: ``__mongodb_heartbeats``
425
430
| **Accepted Values**: A valid Kafka topic name
426
431
@@ -557,6 +562,33 @@ events, and broadcast changes after any collection data is updated.
557
562
database=example
558
563
copy.existing=true
559
564
565
+ You can use the ``copy.existing.namespace.regex`` setting to provide
566
+ a regular expression that matches specific collections by their namespace
567
+ (database and collection name, separated with a "." character).
568
+
569
+ .. code-block:: properties
570
+
571
+ copy.existing.namespace.regex=stats\.page.*
572
+ copy.existing=true
573
+
574
+ The configuration shown above matches and copies existing data from all
575
+ collections within the ``stats`` database. E.g. it matches
576
+ ``stats.pageViews`` or ``stats.pageAverageRevenue``, but not
577
+ ``stat.pageViews`` or ``stats.uniquePages``.
578
+
579
+ You can use the ``copy.existing.pipeline`` setting to provide a MongoDB
580
+ aggregation pipeline expression that matches specific documents to include
581
+ in the copy operation.
582
+
583
+ .. code-block:: properties
584
+
585
+ copy.existing.pipeline=[ { $match: { totalUniqueViews: { $gte: 5000 }} ]
586
+ copy.existing=true
587
+
588
+ The configuration shown above matches and copies existing documents that
589
+ contain a field named ``totalUniqueViews`` with a value of ``5000`` or
590
+ greater.
591
+
560
592
How to Recover from An Invalid Resume Token
561
593
-------------------------------------------
562
594
0 commit comments