Skip to content

Commit eb4bd2e

Browse files
author
Dave Cuthbert
authored
DOCS-14974 basic support for sharded v0.9.0 (#57)
* DOCS-14794 support for sharded clusters * Staging fixes * Staging fixes * Staging fixes * Review feedback * Review feedback * Review feedback * Review feedback * Intro rewrite * Staging updates * Review feedback * fake commit * DOCS-14974 basic support for sharded * Review feedback * Review feedback * Review feedback * Typo * Review feedback * Review feedback * Staging fixes * Staging fixes * review feedback
1 parent 413c6d8 commit eb4bd2e

File tree

3 files changed

+288
-0
lines changed

3 files changed

+288
-0
lines changed

‎snooty.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,7 @@ toc_landing_pages = ["/quickstart",
1717
"reference/reference",
1818
"/connecting",
1919
"/using-mongosync",
20+
"/multiple-mongosyncs",
2021
"/release-notes/release-notes",
2122
"/faq"
2223
]

‎source/index.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,7 @@ common questions users have asked about ``mongosync``.
3838
/quickstart
3939
/installation
4040
/connecting
41+
/multiple-mongosyncs
4142
/reference
4243
/release-notes
4344
/faq

‎source/multiple-mongosyncs.txt

Lines changed: 286 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,286 @@
1+
.. _c2c-sharded-clusters:
2+
3+
=====================================
4+
Use ``mongosync`` on Sharded Clusters
5+
=====================================
6+
7+
.. include:: /includes/preview-warning
8+
9+
.. default-domain:: mongodb
10+
11+
.. contents:: On this page
12+
:local:
13+
:backlinks: none
14+
:depth: 1
15+
:class: twocols
16+
17+
There are two ways to synchronize :ref:`sharded clusters
18+
<sharded-cluster>`. You can use either one ``monogosync`` or several
19+
``monogosync`` instances. For best performance with large or heavily
20+
loaded clusters, use multiple ``monogosync`` instances, one
21+
``monogosync`` for each shard in the cluster.
22+
23+
.. _c2c-sharded-config-single:
24+
25+
Configure a Single ``mongosync`` Instance
26+
-----------------------------------------
27+
28+
To configure a single ``mongosync``, follow the :ref:`connection
29+
instructions <c2c-conn-top-level>` for your cluster architecture to
30+
connect to the :binary:`mongos` instance in your cluster.
31+
32+
When you connect a single ``mongosync`` to a sharded cluster do not use
33+
the :urioption:`replicaSet` option or the :option:`id <mongosync --id>`
34+
option.
35+
36+
The rest of this page addresses cluster to cluster synchronization
37+
using multiple ``monogosync`` instances.
38+
39+
.. _c2c-sharded-config:
40+
41+
Configure Multiple ``mongosync`` Instances
42+
------------------------------------------
43+
44+
To configure multiple ``mongosync`` instances:
45+
46+
#. :ref:`Verify cluster configuration <c2c-shard-config-verify>`
47+
#. :ref:`Determine the shard IDs <c2c-shard-config-determine>`
48+
#. :ref:`Connect the instances <c2c-shard-config-connect>`
49+
50+
.. procedure::
51+
:style: normal
52+
53+
.. step:: Verify cluster configuration
54+
55+
.. _c2c-shard-config-verify:
56+
57+
The source cluster and destination cluster must have the same
58+
number of shards.
59+
60+
.. step:: Determine the shard IDs
61+
62+
.. _c2c-shard-config-determine:
63+
64+
To get the shard IDs, connect to the source cluster
65+
:binary:`mongos` and run the :dbcommand:`listShards` command.
66+
67+
.. code-block:: javascript
68+
69+
db.adminCommand( { listShards: 1 } )
70+
71+
The information is in the ``shards`` array.
72+
73+
.. code-block:: javascript
74+
:copyable: false
75+
:emphasize-lines: 3,9
76+
77+
shards: [
78+
{
79+
_id: 'shard01',
80+
host: 'shard01/localhost:27501,localhost:27502,localhost:27503',
81+
state: 1,
82+
topologyTime: Timestamp({ t: 1656612236, i: 2 })
83+
},
84+
{
85+
_id: 'shard02',
86+
host: 'shard02/localhost:27504,localhost:27505,localhost:27506',
87+
state: 1,
88+
topologyTime: Timestamp({ t: 1656612240, i: 4 })
89+
}
90+
]
91+
92+
.. step:: Connect the ``mongosync`` instances
93+
94+
.. _c2c-shard-config-connect:
95+
96+
These instructions use a generic connection string. To modify the
97+
connection string for your cluster architecture, refer to the
98+
architecture specific :ref:`connection details <c2c-connecting>`.
99+
100+
.. tip::
101+
102+
A single host server can run multiple ``mongosync`` instances. To
103+
improve performance, run ``mongosync`` on multiple host servers.
104+
105+
Each ``mongosync`` instance:
106+
107+
- Connects to the same ``mongos`` instance in the source cluster.
108+
- Connects to the same ``mongos`` instance in the destination
109+
cluster.
110+
- Connects to a single shard in the source cluster.
111+
- Specifies a unique port to use for during synchronization.
112+
Consider designating a range of ports to simplify scripting
113+
{+c2c-product-name+} operations.
114+
115+
Run the first ``mongosync`` instance:
116+
117+
.. code-block:: javascript
118+
119+
mongosync \
120+
--cluster0 "mongodb://user:password@cluster0host:27500" \
121+
--cluster1 "mongodb://user:password@cluster1host:27500" \
122+
--id shard01 --port 27601
123+
124+
Run a new ``mongosync`` instance for each shard in the source cluster.
125+
Edit the ``--id`` and ``--port`` fields for each additional
126+
``mongosync`` instance.
127+
128+
.. code-block:: javascript
129+
:emphasize-lines: 4
130+
131+
mongosync \
132+
--cluster0 "mongodb://user:password@cluster0host:27500" \
133+
--cluster1 "mongodb://user:password@cluster1host:27500" \
134+
--id shard02 --port 27602
135+
136+
.. _c2c-sharded-start:
137+
138+
Start Multiple ``mongosync`` Instances
139+
--------------------------------------
140+
141+
Use ``curl`` or another HTTP client to issue the :ref:`start
142+
<c2c-api-start>` command to each of the ``mongosync`` instances.
143+
144+
.. code-block:: shell
145+
146+
curl monogsync01Host:27601/api/v1/start -XPOST --data \
147+
'{ "source": "cluster0", "destination": "cluster1", \
148+
"reversible": false, "enableUserWriteBlocking": false }'
149+
150+
curl monogsync02Host:27602/api/v1/start -XPOST --data \
151+
'{ "source": "cluster0", "destination": "cluster1", \
152+
"reversible": false, "enableUserWriteBlocking": false }'
153+
154+
The ``start`` command options must be the same for all of the ``mongosync``
155+
instances.
156+
157+
.. _c2c-sharded-progress:
158+
159+
Check Progress
160+
--------------
161+
162+
The ``mongosync`` instances do not aggregate progress information
163+
across shards. To review synchronization progress for a particular
164+
shard, use ``curl`` or another HTTP client to issue the
165+
:ref:`progress <c2c-api-progress>` command to the ``mongosync``
166+
instance synchronizing that shard.
167+
168+
.. code-block:: shell
169+
170+
curl monogsync02Host:27602/api/v1/progress -XGET
171+
172+
This command checks the progress of the ``mongosync`` instance that is
173+
running on ``monogsync02Host`` and using ``port 27602`` for
174+
synchronization. To check progress on other shards, update the host and
175+
port number then repeat the API call to each ``mongosync`` instance.
176+
177+
.. _c2c-sharded-pause:
178+
179+
Pause a ``mongosync`` Instance
180+
------------------------------
181+
182+
The :ref:`pause <c2c-api-pause>` command will temporarily halt the
183+
synchronization process on a single shard. It does not pause any other
184+
``mongosync`` instances that may be running. Use ``curl`` or another
185+
HTTP client to issue the ``pause`` command to a ``mongosync`` instance.
186+
187+
.. code-block:: shell
188+
189+
curl monogsync01Host:27601/api/v1/pause -XPOST --data '{}'
190+
191+
This command pauses the ``mongosync`` instance that is running on
192+
``monogsync01Host`` and using ``port 27601`` for synchronization. To
193+
pause synchronization on other shards, update the host and port number
194+
then repeat the API call to each ``mongosync`` instance.
195+
196+
.. _c2c-sharded-resume:
197+
198+
Resume Synchronization
199+
----------------------
200+
201+
If one or more ``mongosync`` instances are paused, you can use the
202+
:ref:`resume <c2c-api-resume>` command to resume synchronizing. Run a
203+
separate ``resume`` command against each paused ``mongosync`` instance
204+
to continue synchronizing.
205+
206+
Use ``curl`` or another HTTP client to issue the :ref:`resume
207+
<c2c-api-resume>` command to each ``mongosync`` instance.
208+
209+
.. code-block:: shell
210+
211+
curl monogsync01Host:27601/api/v1/resume -XPOST --data '{}'
212+
213+
This command resumes synchronization on the ``mongosync`` instance that
214+
is running on ``monogsync01Host`` and using ``port 27601``. To
215+
resume synchronization on other shards, update the host and port number
216+
then repeat the API call to each ``mongosync`` instance.
217+
218+
.. _c2c-sharded-commit:
219+
220+
Commit Synchronization From Multiple ``mongosync`` Instances
221+
-------------------------------------------------------------
222+
223+
When you want to complete synchronization, issue the :ref:`progress
224+
<c2c-api-progress>` command and check the values for ``canCommit``
225+
and ``lagTimeSeconds``.
226+
227+
To minimize write blocking on the source cluster, you should only run
228+
the ``commit`` command when the :ref:`lagTimeSeconds
229+
<c2c-api-progress>` value is small enough for your application.
230+
231+
If the ``lagTimeSeconds`` value is small enough, and ``canCommit`` is
232+
``true``, issue the :ref:`commit <c2c-api-commit>` command to commit
233+
synchronization. Repeat the process on all of the ``mongosync``
234+
instances.
235+
236+
The ``commit`` operation is blocking. The ``commit`` command will not
237+
return until ``commit`` has been called on every ``mongosync``
238+
instance.
239+
240+
.. code-block:: shell
241+
242+
// Check progress
243+
curl monogsync01Host:27601/api/v1/progress -XGET
244+
245+
// Commit
246+
curl monogsync01Host:27601/api/v1/commit -XPOST --data '{}'
247+
248+
These commands only check progress and commit synchronization for the
249+
``mongosync`` instance that is running on ``monogsync01Host`` and using
250+
``port 27601``. To synchronize all of the shards, make additional calls
251+
to ``progress`` and ``commit`` on any other ``mongosync`` instances
252+
that may be running.
253+
254+
.. _c2c-sharded-reverse:
255+
256+
Reverse the Synchronization Direction
257+
-------------------------------------
258+
259+
To reverse synchronization so that the original destination cluster
260+
acts as the source cluster:
261+
262+
- If you have not already done so, issue the :ref:`commit
263+
<c2c-api-commit>` command to each ``mongosync`` instance and wait
264+
until all of the commits to finish.
265+
- Issue the :ref:`reverse <c2c-api-reverse>` command to each
266+
``mongosync`` instance.
267+
268+
The ``reverse`` operation is blocking. The ``reverse`` command will not
269+
return until ``reverse`` has been called on every ``mongosync``
270+
instance.
271+
272+
.. code-block:: shell
273+
274+
curl monogsync01Host:27601/api/v1/reverse -XPOST --data '{}'
275+
276+
This command reverses synchronization on the ``mongosync``
277+
instance that is running on ``monogsync01Host`` and using ``port
278+
27601``. Make additional calls to ``reverse`` on any other
279+
``mongosync`` instances that may be running.
280+
281+
.. note::
282+
283+
Reverse synchronization is only possible if ``reversible`` and
284+
``enableUserWriteBlocking`` are both set to ``true`` when the
285+
:ref:`start API <c2c-api-start>` initiates ``mongosync``.
286+

0 commit comments

Comments
 (0)