Skip to content

Commit 450c92b

Browse files
sonderdonk-mdbsonderdonk-mdb
authored
DOCSP-41105 "About mongosync" page (#347)
* DOCSP-41105 second draft * DOCSP-41105-2 * DOCSP-41105-2
1 parent 550b5de commit 450c92b

File tree

9 files changed

+236
-40
lines changed

9 files changed

+236
-40
lines changed

‎source/about-mongosync.txt

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
.. _about-mongosync:
2+
3+
====================
4+
About ``mongosync``
5+
====================
6+
7+
.. default-domain:: mongodb
8+
9+
.. contents:: On this page
10+
:local:
11+
:backlinks: none
12+
:depth: 2
13+
:class: singlecol
14+
15+
The ``mongosync`` binary is the primary process used in {+c2c-product-name+}.
16+
``mongosync`` migrates data from a source cluster to a destination cluster and
17+
keeps the clusters in continuous sync until you
18+
:ref:`finalize <c2c-about-finalizing>` the sync.
19+
20+
You can use ``mongosync`` to create
21+
dedicated analytics, development, or testing clusters that mirror your
22+
production environment.
23+
24+
``mongosync`` keeps track of its current actions through
25+
:ref:`states <c2c-states>`. ``mongosync`` enters different states depending on
26+
the requests it receives. The current ``mongosync`` state determines which API
27+
operations you can run.
28+
29+
.. include:: /includes/fact-state-transitions.rst
30+
31+
To view the current state of ``mongosync``, use the :ref:`/progress
32+
<c2c-api-progress>` endpoint.
33+
34+
The following diagram illustrates the relationship between ``mongosync`` states
35+
and API operations:
36+
37+
.. figure:: /images/mongosync.drawio.bakedsvg.svg
38+
:alt: Diagram of relationship between ``mongosync`` states and API operations
39+
:figwidth: 600px
40+
41+
Details
42+
-------
43+
44+
Connect Source and Destination Clusters
45+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
46+
47+
You must create an initial connection to the source and destination
48+
clusters before ``mongosync`` can start to sync data. For more information, see
49+
:ref:`c2c-connecting`.
50+
51+
When ``mongosync`` connects to the source and destination clusters, the
52+
synchronization process enters the :ref:`IDLE <c2c-state-idle>` state.
53+
54+
Start Sync
55+
~~~~~~~~~~
56+
57+
To start the synchronization process, call the :ref:`start <c2c-api-start>`
58+
endpoint.
59+
60+
The ``start`` endpoint begins the ``RUNNING`` state, which is when ``mongosync``
61+
syncs data from the source cluster to the destination cluster. If you
62+
make subsequent writes to the source cluster, ``mongosync`` applies the
63+
operations to the destination cluster.
64+
65+
For every source collection that syncs,
66+
``mongosync`` creates a corresponding collection on the destination.
67+
68+
``mongosync`` temporarily alters some of the replicated collection
69+
characteristics in the destination cluster. It changes these destination cluster
70+
characteristics back to the source cluster characteristics during the ``COMMIT``
71+
state. For more information, see
72+
:ref:`Collection Characteristic Changes <c2c-collection-changes>`.
73+
74+
The ``RUNNING`` state includes two phases:
75+
76+
- :ref:`Collection copy <c2c-collection-copy>`
77+
78+
- :ref:`Change event application <c2c-cea>`
79+
80+
.. _c2c-collection-copy:
81+
82+
Collection Copy
83+
'''''''''''''''
84+
85+
During the collection copy phase, ``mongosync`` executes the initial data
86+
transfer from the source cluster to the destination
87+
cluster. ``mongosync`` performs this data transfer by splitting the
88+
source collections into partitions and copying the partitions to their
89+
corresponding destination collections.
90+
91+
.. _c2c-cea:
92+
93+
Change Event Application
94+
''''''''''''''''''''''''
95+
96+
The change event application (CEA) phase starts after the collection copy phase
97+
ends.
98+
99+
During the CEA phase, ``mongosync`` continuously
100+
updates the destination cluster when you make changes to the source cluster.
101+
102+
To make continuous updates to the destination cluster, ``mongosync`` opens a
103+
:ref:`change stream <changeStreams>` on the source cluster and applies the
104+
:ref:`change events <change-events>` it receives to the destination cluster.
105+
106+
When the CEA phase begins, ``mongosync`` applies the change events
107+
that are received during the collection copy phase before processing the
108+
events that are received during the CEA phase.
109+
110+
Pause Sync
111+
~~~~~~~~~~
112+
113+
To pause the sync process and enter the :ref:`PAUSED <c2c-state-paused>` state,
114+
call the :ref:`pause <c2c-api-pause>` endpoint.
115+
116+
To resume a paused sync process and return ``mongosync`` to the
117+
``RUNNING`` state, call the :ref:`resume <c2c-api-resume>` endpoint.
118+
119+
.. _c2c-about-finalizing:
120+
121+
Finalize Sync
122+
~~~~~~~~~~~~~
123+
124+
To finalize the sync between the source and destination clusters,
125+
call the :ref:`commit <c2c-api-commit>` endpoint.
126+
127+
The ``commit`` endpoint starts the :ref:`COMMITTING <c2c-state-committing>`
128+
state, which is when ``mongosync`` stops continuous sync between the source and
129+
destination clusters.
130+
131+
When the ``COMMITTING`` state begins, ``mongosync`` retrieves the
132+
commit timestamp, which is the time of the most recent operation on the source
133+
cluster.
134+
135+
If there are no errors, ``mongosync`` sends a success message and exits CEA by
136+
processing events up until the commit timestamp. ``mongosync`` also restores
137+
collection characteristics that ``mongosync`` temporarily alters during
138+
synchronization.
139+
140+
When the above process is completed, ``mongosync`` enters the
141+
:ref:`COMMITTED <c2c-state-committed>` state. The ``COMMITTED`` state indicates
142+
that the :ref:`cutover process <c2c-cutover-process>` is finished.
143+
144+
.. _c2c-about-reversing:
145+
146+
Reverse Sync
147+
~~~~~~~~~~~~
148+
149+
To reverse the direction of a committed sync operation, call the
150+
:ref:`reverse <c2c-api-reverse>` endpoint.
151+
152+
The ``reverse`` endpoint starts the :ref:`REVERSING <c2c-state-reversing>`
153+
state. ``mongosync`` swaps the source and destination clusters and resumes
154+
applying change events.
155+
156+
If the reverse sync is successful, ``mongosync`` enters the ``RUNNING`` state.
157+
The synchronization continues in the reverse direction from the original
158+
sync job.
159+
160+
Learn More
161+
----------
162+
163+
To learn more about ``mongosync``, see:
164+
165+
- :ref:`mongosync <c2c-mongosync>`
166+
- :ref:`mongosync States <c2c-states>`

‎source/images/mongosync.drawio.bakedsvg.svg

Lines changed: 1 addition & 0 deletions
Loading

‎source/images/mongosync.drawio.svg

Lines changed: 4 additions & 0 deletions
Loading

‎source/includes/fact-state-transitions.rst

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
.. note::
2+
3+
Most state transitions happen as the result of an API call. The exceptions
4+
are:
5+
6+
- The transition from ``COMMITTING`` to ``COMMITTED``. For more information,
7+
see :ref:`Finalizing Sync <c2c-about-finalizing>`.
8+
9+
- The transition from ``REVERSING`` to ``RUNNING``. For more information,
10+
see :ref:`Reversing Sync <c2c-about-reversing>`.
11+

‎source/index.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,8 @@ and data residency compliance.
2121
In addition to continuous data synchronization, ``mongosync`` can also
2222
facilitate a one time data migration between clusters.
2323

24+
For an overview of ``mongosync``, see :ref:`about-mongosync`.
25+
2426
To get started with ``mongosync``, refer to the :ref:`Quick Start Guide
2527
<c2c-quickstart>`. For more detailed information, refer to the
2628
:ref:`c2c-install` or :ref:`c2c-connecting` page that best fits your
@@ -36,6 +38,7 @@ common questions users have asked about ``mongosync``.
3638
:titlesonly:
3739

3840
Quickstart </quickstart>
41+
/about-mongosync
3942
/installation
4043
/connecting
4144
/multiple-mongosyncs

‎source/reference.txt

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@ Reference
33
=========
44

55
:ref:`c2c-mongosync`
6-
The primary process used in {+c2c-product-name+}.
6+
The options and behavior in {+c2c-product-name+}.
77

88
:ref:`c2c-config`
99
Configure :program:`mongosync` instances at startup using a

‎source/reference/api/commit.txt

Lines changed: 46 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -16,45 +16,7 @@ Description
1616
-----------
1717

1818
Finalizes the sync between the source cluster and the destination
19-
cluster and stops continuous sync between clusters. ``commit`` also
20-
restores collection characteristics that ``mongosync`` temporarily
21-
alters during synchronization.
22-
23-
The following table illustrates the impact of ``commit`` on collection
24-
characteristics that ``mongosync`` alters during the synchronization process.
25-
26-
.. list-table::
27-
:header-rows: 1
28-
29-
* - Collection Characteristic
30-
- Impact of ``commit``
31-
* - :ref:`Unique Indexes <c2c-dr-unique>`
32-
- ``commit`` resets unique indexes that ``mongosync`` replicated as
33-
non-unique on the destination cluster.
34-
35-
* - :ref:`TTL Indexes <c2c-dr-ttl>`
36-
- ``commit`` resets ``expireAfterSeconds`` which ``mongosync`` set
37-
to the value of ``MAX_INT`` on the destination cluster.
38-
39-
* - :ref:`Hidden Indexes <c2c-dr-hidden>`
40-
- ``commit`` resets hidden indexes that ``mongosync``
41-
replicated as non-hidden on the destination cluster.
42-
43-
* - :ref:`Write Blocking <c2c-dr-write-blocking>`
44-
- If you enable write-blocking, ``mongosync`` blocks writes:
45-
46-
- On the destination cluster during sync.
47-
- On the source cluster when ``commit`` is received.
48-
49-
.. seealso::
50-
51-
- :ref:`c2c-failure-recovery`
52-
- :ref:`c2c-write-blocking`
53-
54-
* - :ref:`Capped Collections <c2c-dr-capped-collections>`
55-
- ``commit`` resets the required maximum size of capped collections
56-
which ``mongosync`` set to the maximum allowable size on the
57-
destination cluster.
19+
cluster and stops continuous sync between clusters.
5820

5921
Requirements
6022
------------
@@ -170,6 +132,51 @@ Data Verification
170132

171133
.. include:: /includes/fact-verification
172134

135+
.. _c2c-collection-changes:
136+
137+
Collection Characteristic Changes
138+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
139+
140+
``commit`` restores collection characteristics that ``mongosync`` temporarily
141+
alters during synchronization.
142+
143+
The following table illustrates the impact of ``commit`` on collection
144+
characteristics that ``mongosync`` alters during the synchronization process.
145+
146+
.. list-table::
147+
:header-rows: 1
148+
149+
* - Collection Characteristic
150+
- Impact of ``commit``
151+
* - :ref:`Unique Indexes <c2c-dr-unique>`
152+
- ``commit`` resets unique indexes that ``mongosync`` replicated as
153+
non-unique on the destination cluster.
154+
155+
* - :ref:`TTL Indexes <c2c-dr-ttl>`
156+
- ``commit`` resets ``expireAfterSeconds`` which ``mongosync`` set
157+
to the value of ``MAX_INT`` on the destination cluster.
158+
159+
* - :ref:`Hidden Indexes <c2c-dr-hidden>`
160+
- ``commit`` resets hidden indexes that ``mongosync``
161+
replicated as non-hidden on the destination cluster.
162+
163+
* - :ref:`Write Blocking <c2c-dr-write-blocking>`
164+
- If you enable write-blocking, ``mongosync`` blocks writes:
165+
166+
- On the destination cluster during sync.
167+
- On the source cluster when ``commit`` is received.
168+
169+
.. seealso::
170+
171+
- :ref:`c2c-failure-recovery`
172+
- :ref:`c2c-write-blocking`
173+
174+
* - :ref:`Capped Collections <c2c-dr-capped-collections>`
175+
- ``commit`` resets the required maximum size of capped collections
176+
which ``mongosync`` set to the maximum allowable size on the
177+
destination cluster.
178+
179+
173180
Endpoint Protection
174181
~~~~~~~~~~~~~~~~~~~
175182

‎source/reference/mongosync-states.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,8 @@ receives. ``mongosync`` can only be in a single state at a given time.
1717
The current ``mongosync`` states determines which API operations you can
1818
run.
1919

20+
.. include:: /includes/fact-state-transitions.rst
21+
2022
This page describes ``mongosync`` states.
2123

2224
View the Current State

‎source/reference/mongosync.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -206,6 +206,8 @@ Even if documents move between source shards during replication,
206206
{+c2c-product-name+} maintains the :term:`eventual consistency`
207207
guarantee on the destination cluster.
208208

209+
.. _c2c-mongosync-ddl:
210+
209211
Data Definition Language (DDL) Operations
210212
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
211213

0 commit comments

Comments
 (0)