Skip to content

DOCS-11828 document 'startAfter' option for $changeStream #3434

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Oct 3, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 36 additions & 5 deletions source/changeStreams.txt
Original file line number Diff line number Diff line change
Expand Up @@ -491,11 +491,19 @@ document.
Resume a Change Stream
----------------------

Change streams are resumable by specifying a ``resumeAfter`` token when
opening the cursor. For the ``resumeAfter`` token, use the ``_id``
value of the :ref:`change stream event document
<change-stream-output>`. Passing the ``_id`` value to the change stream
attempts to resume notifications starting after the specified operation.
Change streams are resumable by specifying a resume token to either
:ref:`resumeAfter <change-stream-resume-after>` or
:ref:`startAfter <change-stream-start-after>` when opening the cursor.

.. _change-stream-resume-after:

``resumeAfter`` for Change Streams
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can resume a change stream after a specific event by passing a resume token
to ``resumeAfter`` when opening the cursor. For the resume token, use the
``_id`` value of the :ref:`change stream event document <change-stream-output>`.
See :ref:`change-stream-resume-token` for more information on the resume token.

.. important::

Expand Down Expand Up @@ -647,6 +655,29 @@ attempts to resume notifications starting after the specified operation.
:start-after: Start Changestream Example 3
:end-before: End Changestream Example 3

.. _change-stream-start-after:

``startAfter`` for Change Streams
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. versionadded:: 4.2

You can start a new change stream after a specific event by passing a resume
token to ``startAfter`` when opening the cursor. Unlike
:ref:`resumeAfter <change-stream-resume-after>`, ``startAfter`` can
resume notifications after an :ref:`invalidate event <change-event-invalidate>`
by creating a new change stream. For the resume token, use the ``_id`` value of
the :ref:`change stream event document <change-stream-output>`. See
:ref:`change-stream-resume-token` for more information on the resume token.

.. important::

- The oplog must have enough history to locate the operation
associated with the token or the timestamp, if the timestamp is in
the past.

.. _change-stream-resume-token:

Resume Token and ``featureCompatibilityVersion``
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down
36 changes: 30 additions & 6 deletions source/includes/apiargs-method-Mongo.watch-options.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,40 @@ description: |
``_id`` field. Pass the *entire* ``_id`` field of the change event
document that represents the operation you want to resume after.

``resumeAfter`` is mutually exclusive with ``startAtOperationTime``.
``resumeAfter`` is mutually exclusive with ``startAfter`` and
``startAtOperationTime``.

optional: true
type: document
operation: Mongo.watch
interface: method
arg_name: field
---
name: fullDocument
name: startAfter
position: 2
description: |

Directs :method:`Mongo.watch()` to attempt starting a new change
stream after the operation specified in the resume token. Allows
notifications to resume after an invalidate event.

Each change stream event document includes a resume token as the
``_id`` field. Pass the *entire* ``_id`` field of the change event
document that represents the operation you want to resume after.

``startAfter`` is mutually exclusive with ``resumeAfter`` and
``startAtOperationTime``.

.. versionadded:: 4.2

optional: true
type: document
operation: Mongo.watch
interface: method
arg_name: field
---
name: fullDocument
position: 3
description: |

By default, :method:`Mongo.watch()` returns the delta of
Expand All @@ -39,7 +63,7 @@ interface: method
arg_name: field
---
name: batchSize
position: 3
position: 4
description: |
Specifies the maximum number of change events to return in each
batch of the response from the MongoDB cluster.
Expand All @@ -52,7 +76,7 @@ interface: method
arg_name: field
---
name: maxAwaitTimeMS
position: 4
position: 5
description: |

The maximum amount of time in milliseconds the server waits for new
Expand All @@ -68,7 +92,7 @@ interface: method
arg_name: field
---
name: collation
position: 5
position: 6
description: |
Pass a :ref:`collation document <collation-document-fields>`
to specify a :doc:`collation </reference/collation>` for the
Expand All @@ -80,7 +104,7 @@ interface: method
arg_name: field
---
name: startAtOperationTime
position: 6
position: 7
source:
file: apiargs-method-db.watch-options.yaml
ref: startAtOperationTime
Expand Down
36 changes: 30 additions & 6 deletions source/includes/apiargs-method-db.collection.watch-options.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,40 @@ description: |
``_id`` field. Pass the *entire* ``_id`` field of the change event
document that represents the operation you want to resume after.

``resumeAfter`` is mutually exclusive with ``startAtOperationTime``.
``resumeAfter`` is mutually exclusive with ``startAfter`` and
``startAtOperationTime``.

optional: true
type: document
operation: db.collection.watch
interface: method
arg_name: field
---
name: fullDocument
name: startAfter
position: 2
description: |

Directs :method:`~db.collection.watch` to attempt starting a new
change stream after the operation specified in the resume token.
Allows notifications to resume after an invalidate event.

Each change stream event document includes a resume token as the
``_id`` field. Pass the *entire* ``_id`` field of the change event
document that represents the operation you want to resume after.

``startAfter`` is mutually exclusive with ``resumeAfter`` and
``startAtOperationTime``.

.. versionadded:: 4.2

optional: true
type: document
operation: db.collection.watch
interface: method
arg_name: field
---
name: fullDocument
position: 3
description: |

By default, :method:`~db.collection.watch()` returns the delta of
Expand All @@ -39,7 +63,7 @@ interface: method
arg_name: field
---
name: batchSize
position: 3
position: 4
description: |
Specifies the maximum number of change events to return in each
batch of the response from the MongoDB cluster.
Expand All @@ -52,7 +76,7 @@ interface: method
arg_name: field
---
name: maxAwaitTimeMS
position: 4
position: 5
description: |

The maximum amount of time in milliseconds the server waits for new
Expand All @@ -68,7 +92,7 @@ interface: method
arg_name: field
---
name: collation
position: 5
position: 6
description: |
Pass a :ref:`collation document <collation-document-fields>`
to specify a :doc:`collation </reference/collation>` for the
Expand All @@ -80,7 +104,7 @@ interface: method
arg_name: field
---
name: startAtOperationTime
position: 6
position: 7
source:
file: apiargs-method-db.watch-options.yaml
ref: startAtOperationTime
Expand Down
28 changes: 26 additions & 2 deletions source/includes/apiargs-method-db.watch-options.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,30 @@ description: |
``_id`` field. Pass the *entire* ``_id`` field of the change event
document that represents the operation you want to resume after.

``resumeAfter`` is mutually exclusive with ``startAtOperationTime``.
``resumeAfter`` is mutually exclusive with ``startAfter`` and
``startAtOperationTime``.

optional: true
type: document
operation: db.watch
interface: method
arg_name: field
---
name: startAfter
description: |

Directs :method:`db.watch()` to attempt starting a new change stream
after the operation specified in the resume token. Allows
notifications to resume after an invalidate event.

Each change stream event document includes a resume token as the
``_id`` field. Pass the *entire* ``_id`` field of the change event
document that represents the operation you want to resume after.

``startAfter`` is mutually exclusive with ``resumeAfter`` and
``startAtOperationTime``.

.. versionadded:: 4.2

optional: true
type: document
Expand Down Expand Up @@ -82,7 +105,8 @@ description: |
check the time range of the oplog, see
:method:`rs.printReplicationInfo()`.

``startAtOperationTime`` is mutually exclusive with ``resumeAfter``.
``startAtOperationTime`` is mutually exclusive with ``resumeAfter``
and ``startAfter``.

optional: true
type: Timestamp
Expand Down
13 changes: 8 additions & 5 deletions source/includes/extracts-changestream.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,9 @@ content: |
|watchmethod| uses information stored in the oplog to produce the
change event description and generate a resume token associated to
that operation. If the operation identified by the resume token
passed to the ``resumeAfter`` option has already dropped off the
:term:`oplog`, |watchmethod| cannot resume the change stream.
passed to the ``resumeAfter`` or ``startAfter`` option has already
dropped off the :term:`oplog`, |watchmethod| cannot resume the
change stream.

See :ref:`change-stream-resume` for more information on
resuming a change stream.
Expand Down Expand Up @@ -59,9 +60,11 @@ content: |
ref: changestream-invalid-events
content: |

You cannot resume a change stream after an :ref:`invalidate event
<change-event-invalidate>` (for example, a collection drop or rename)
closes the stream.
You cannot use ``resumeAfter`` to resume a change stream after an
:ref:`invalidate event <change-event-invalidate>` (for example, a collection
drop or rename) closes the stream. Starting in MongoDB 4.2, you can use
:ref:`startAfter <change-stream-start-after>` to start a new change
stream after an :ref:`invalidate event <change-event-invalidate>`.

---
ref: changestream-cursor-open
Expand Down
3 changes: 2 additions & 1 deletion source/reference/method/db.collection.watch.txt
Original file line number Diff line number Diff line change
Expand Up @@ -190,7 +190,8 @@ Resuming a Change Stream
Every document returned by a change stream cursor includes a resume
token as the ``_id`` field. To resume a change stream, pass the entire
``_id`` document of the change event you want to resume from to
the ``resumeAfter`` option of :method:`~db.collection.watch()`.
either the ``resumeAfter`` or ``startAfter`` option of
:method:`~db.collection.watch()`.

The following operation resumes a change stream cursor against the
``data.sensors`` collection using a resume token. This
Expand Down
9 changes: 9 additions & 0 deletions source/release-notes/4.2.txt
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,15 @@ An example of the old syntax is:

{ $out: "stock" }

``startAfter`` Option for Change Streams
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

MongoDB 4.2 adds ``startAfter`` as an option for :ref:`changeStreams`, which
starts a new change stream after the event indicated by a resume token. With
this option, you can start a change stream from an
:ref:`invalidate event <change-event-invalidate>`, thereby guaranteeing no
missed notifications after the previous stream was invalidated.

Changes Affecting Compatibility
-------------------------------

Expand Down