Skip to content

DOCS-94 DOCS-95 migrate sharded cluster to new hardware #1257

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions bin/builddata/htaccess.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2865,6 +2865,13 @@ redirect-path: '/MongoDB-manual-v2.2.tar.gz'
url-base: '/mongodb-manual-v2.2.tar.gz'
type: 'redirect'
code: 301
outputs:
- 'v2.2'
---
redirect-path: '/tutorial/migrate-sharded-cluster-to-new-hardware'
url-base: '/administration/sharded-clusters'
type: 'redirect'
code: 303
outputs:
- 'v2.2'
...
5 changes: 5 additions & 0 deletions source/includes/tip-hostnames.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
.. tip:: To avoid downtime, give each config server a logical DNS name
(unrelated to the server's physical or virtual hostname). Without
logical DNS names, moving or renaming a config server requires
shutting down every :program:`mongod` and :program:`mongos` instance
in the sharded cluster.
5 changes: 5 additions & 0 deletions source/includes/toc-sharded-cluster-maintenance.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,11 @@ description: |
Replaces a config server that has become inoperable. This procedure
assumes that the hostname does not change.
---
file: /tutorial/migrate-sharded-cluster-to-new-hardware
description: |
Migrate a sharded cluster to a different hardware system, for example,
when moving a pre-production environment to production.
---
file: /tutorial/backup-sharded-cluster-metadata
description: |
Create a backup of a sharded cluster's metadata while keeping the
Expand Down
1 change: 1 addition & 0 deletions source/tutorial.txt
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ Sharding
- :doc:`/tutorial/migrate-config-servers-with-same-hostname`
- :doc:`/tutorial/migrate-config-servers-with-different-hostnames`
- :doc:`/tutorial/replace-config-server`
- :doc:`/tutorial/migrate-sharded-cluster-to-new-hardware`
- :doc:`/tutorial/backup-sharded-cluster-metadata`
- :doc:`/tutorial/backup-small-sharded-cluster-with-mongodump`
- :doc:`/tutorial/backup-sharded-cluster-with-filesystem-snapshots`
Expand Down
10 changes: 2 additions & 8 deletions source/tutorial/add-shards-to-shard-cluster.txt
Original file line number Diff line number Diff line change
Expand Up @@ -24,21 +24,15 @@ You interact with a sharded cluster by connecting to a :program:`mongos`
instance.

1. From a :program:`mongo` shell, connect to the :program:`mongos`
instance. Issue a command using the following syntax:

.. code-block:: sh

mongo --host <hostname of machine running mongos> --port <port mongos listens on>

For example, if a :program:`mongos` is accessible at
instance. For example, if a :program:`mongos` is accessible at
``mongos0.example.net`` on port ``27017``, issue the following
command:

.. code-block:: sh

mongo --host mongos0.example.net --port 27017

#. Add each shard to the cluster using the :method:`sh.addShard()`
#. Add a shard to the cluster using the :method:`sh.addShard()`
method, as shown in the examples below. Issue :method:`sh.addShard()`
separately for each shard. If the shard is a replica set, specify the
name of the replica set and specify a member of the set. In
Expand Down
2 changes: 1 addition & 1 deletion source/tutorial/deploy-shard-cluster.txt
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ When you start the :program:`mongos` instance, specify the hostnames of
the three config servers, either in the configuration file or as command
line parameters.

.. tip:: .. include:: /includes/fact-use-cnames-for-config-servers.rst
.. include:: /includes/tip-hostnames.rst

To start a :program:`mongos` instance, issue a command using the following syntax:

Expand Down
85 changes: 40 additions & 45 deletions source/tutorial/manage-sharded-cluster-balancer.txt
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,27 @@ Manage Sharded Cluster Balancer

.. default-domain:: mongodb

This page describes provides common administrative procedures related
This page describes common administrative procedures related
to balancing. For an introduction to balancing, see
:ref:`sharding-balancing`. For lower level information on balancing, see
:ref:`sharding-balancing-internals`.

.. seealso:: :doc:`/tutorial/configure-sharded-cluster-balancer`

Check the Balancer State
------------------------

The following command checks if the balancer is enabled (i.e. that the
balancer is allowed to run). The command does not check if the balancer
is active (i.e. if it is actively balancing chunks).

To see if the balancer is enabled in your :term:`cluster <sharded
cluster>`, issue the following command, which returns a boolean:

.. code-block:: javascript

sh.getBalancerState()

.. _sharding-balancing-check-lock:

Check the Balancer Lock
Expand Down Expand Up @@ -57,15 +71,6 @@ This output confirms that:
has the lock. For version 2.0 and later, the value of an active lock
is ``2``; for earlier versions the value is ``1``.

.. optional::

You can also use the following shell helper, which returns a
boolean to report if the balancer is active:

.. code-block:: javascript

sh.getBalancerState()

.. _sharding-schedule-balancing-window:

Schedule the Balancing Window
Expand Down Expand Up @@ -153,46 +158,36 @@ all migration, use the following procedure:

sh.setBalancerState(false)

.. note::

If a migration is in progress, the system will complete the
in-progress migration before stopping.

#. To later re-enable the balancer, see :ref:`sharding-balancing-re-enable`.
If a migration is in progress, the system will complete the
in-progress migration before stopping.

After disabling, you can use the following
operation in the :program:`mongo` shell to determine if there are
no migrations in progress:
#. To verify that the balancer has stopped, issue the following command,
which returns ``false`` if the balancer is stopped:

.. code-block:: javascript

use config
while( sh.isBalancerRunning() ) {
print("waiting...");
sleep(1000);
}

The :method:`sh.setBalancerState()` helper provides
a wrapper on the following process, which may be useful if you
need to run this operation from a driver that does not have helper
functions:
.. code-block:: javascript

#. Connect to any :program:`mongos` in the cluster using the
:program:`mongo` shell.
sh.getBalancerState()

#. Issue the following command to switch to the :ref:`config-database`:
Optionally, to verify no migrations are in progress after disabling,
issue the following operation in the :program:`mongo` shell:

.. code-block:: javascript

use config
while( sh.isBalancerRunning() ) {
print("waiting...");
sleep(1000);
}

#. Issue the following update to disable the balancer:
.. note::

.. code-block:: javascript
To disable the balancer from a driver that does not have the
:method:`sh.startBalancer()` helper, issue the following command from
the ``config`` database:

db.settings.update( { _id: "balancer" }, { $set : { stopped: true } } , true );
.. code-block:: javascript

#. To later re-enable the balancer, see :ref:`sharding-balancing-re-enable`.
db.settings.update( { _id: "balancer" }, { $set : { stopped: true } } , true )

.. _sharding-balancing-re-enable:
.. _sharding-balancing-enable:
Expand All @@ -208,18 +203,18 @@ re-enable it:

#. Issue one of the following operations to enable the balancer:

- From the :program:`mongo` shell, issue:
From the :program:`mongo` shell, issue:

.. code-block:: javascript
.. code-block:: javascript

sh.setBalancerState(true)
sh.setBalancerState(true)

- From a driver that does not have the :method:`sh.startBalancer()` helper,
issue the following from the ``config`` database:
From a driver that does not have the :method:`sh.startBalancer()` helper,
issue the following from the ``config`` database:

.. code-block:: javascript
.. code-block:: javascript

db.settings.update( { _id: "balancer" }, { $set : { stopped: false } } , true )
db.settings.update( { _id: "balancer" }, { $set : { stopped: false } } , true )

Disable Balancing During Backups
--------------------------------
Expand Down
38 changes: 27 additions & 11 deletions source/tutorial/migrate-config-servers-with-different-hostnames.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,46 +8,62 @@ This procedure migrates a :ref:`config server <sharding-config-server>`
in a :doc:`sharded cluster </core/sharding>`
to a new server that uses a different hostname. Use this procedure only
if the config server *will not* be accessible via the same hostname.

Changing a :ref:`config server's <sharding-config-server>` hostname
**requires downtime** and requires restarting every process in the
sharded cluster.
If possible, avoid changing the hostname so that you can instead use the
procedure to :doc:`migrate a config server and use the same hostname
<migrate-config-servers-with-same-hostname>`.

#. Disable the cluster balancer process temporarily. See
To migrate all the config servers in a cluster, perform this procedure
for each config server separately and migrate the config servers in
reverse order from how they are listed in the :program:`mongos`
instances' :setting:`configdb` string. Start with the last config server
listed in the :setting:`configdb` string.

1. Disable the cluster balancer process temporarily. See
:ref:`sharding-balancing-disable-temporarily` for more information.

#. Shut down the :ref:`config server <sharding-config-server>` you are moving.
#. Shut down the config server.

This renders all config data for the sharded cluster "read only."

#. Copy the contents of :setting:`dbpath` from the old config server to
the new config server.

This will render all config data for your cluster "read only:"
For example, to copy the contents of :setting:`dbpath` to a machine
named ``mongodb.config2.example.net``, you might issue a command
similar to the following:

.. code-block:: sh

rsync -az /data/configdb mongodb.config2.example.net:/data/configdb

#. Start the config instance on the new system. The default invocation
is:
#. Start the config server instance on the new system. The default
invocation is:

.. code-block:: sh

mongod --configsvr

#. Shut down all existing MongoDB processes. This includes:

- all :program:`mongod` instances or :term:`replica sets <replica set>`
- the :program:`mongod` instances or :term:`replica sets <replica set>`
that provide your shards.

- the :program:`mongod` instances that provide your existing
:ref:`config databases <config-database>`.

- all :program:`mongos` instances in your cluster.
- the :program:`mongos` instances.

#. Restart all :program:`mongod` processes that provide the shard
servers.

#. Restart the config instances that have *not* changed hostnames.
#. Update the :setting:`configdb` setting for each :program:`mongos`
instances.

#. Update the :option:`--configdb <mongos --configdb>` parameter (or
:setting:`configdb`) for all :program:`mongos` instances and
restart all :program:`mongos` instances.
#. Restart the :program:`mongos` instances.

#. Re-enable the balancer to allow the cluster to resume normal
balancing operations. See the
Expand Down
31 changes: 21 additions & 10 deletions source/tutorial/migrate-config-servers-with-same-hostname.txt
Original file line number Diff line number Diff line change
Expand Up @@ -8,33 +8,44 @@ This procedure migrates a :ref:`config server <sharding-config-server>`
in a :doc:`sharded cluster </core/sharding>`
to a new system that uses *the same* hostname.

#. Shut down the config server that you are moving.
To migrate all the config servers in a cluster, perform this procedure
for each config server separately and migrate the config servers in
reverse order from how they are listed in the :program:`mongos`
instances' :setting:`configdb` string. Start with the last config server
listed in the :setting:`configdb` string.

This will render all config data for your cluster :ref:`read only
<sharding-config-server>`.
.. start-migrate-config-server-with-same-hostname

#. Shut down the config server.

This renders all config data for the sharded cluster "read only."

#. Change the DNS entry that points to the system that provided the old
config server, so that the *same* hostname points to the new
system.

How you do this depends on how you organize your DNS and
hostname resolution services.

#. Move the entire :setting:`dbpath` file system tree from the old
config server to the new config server. This command, issued on the
old config server system, may resemble the following:
#. Copy the contents of :setting:`dbpath` from the old config server to
the new config server.

For example, to copy the contents of :setting:`dbpath` to a machine
named ``mongodb.config2.example.net``, you might issue a command
similar to the following:

.. code-block:: sh

rsync -az /data/configdb mongo-config0.example.net:/data/configdb
rsync -az /data/configdb mongodb.config2.example.net:/data/configdb

#. Start the config instance on the new system. The default invocation
is:
#. Start the config server instance on the new system. The default
invocation is:

.. code-block:: sh

mongod --configsvr

.. end-migrate-config-server-with-same-hostname

When you start the third config server, your cluster will become
writable and it will be able to create new splits and migrate chunks
as needed.
Loading