Skip to content

2079 when does secondary throttle take effect #1519

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

2079 when does secondary throttle take effect #1519

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
1045613
DOCS-2079 and DOCS-2211: correct secondary throttle explanation
Jan 2, 2014
7b35e93
DOCS-2079: minor clarifications
Jan 2, 2014
855d3b5
DOCS-2079: fixed write concern on secondary throttle from 'majority' …
Jan 3, 2014
59fed5c
DOCS-2079: reorg based on conversation between Sam and Zola
Jan 6, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 58 additions & 9 deletions source/core/sharding-chunk-migration.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,14 +96,63 @@ primary can orphan data from multiple migrations.
Chunk Migration Write Concern
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. versionchanged:: 2.4
Internal operations like chunk migration don't fit exactly into the definitions
of write concern established for user operations. The following explanations
should be taken as approximations.

Write concern for chunk migration depends on whether the balancer or
the :dbcommand:`moveChunk` command operate with ``_secondaryThrottle``
set to ``true`` or ``false``.

Before 2.2.1, chunk migration writes did not wait to replicate to
secondaries before starting the next chunk migration (the equivalent
of ``w: 1``). After all relevant chunks had been written to the
destination shard, the balancer would wait for replication to
a majority of members (the equivalent of ``w: majority``) before
deleting the chunks from the source shard.

This had the effect of creating a replication bottleneck at the end
of migration.

.. versionadded:: 2.2.1

``_secondaryThrottle`` became an option to the balancer and to the
:dbcommand:`moveChunk` command. ``_secondaryThrottle`` requires the
balancer to have a ``{ w: 2 }`` write concern on each delete and
insert operation. This causes the balancer to wait for each chunk
migration to replicate to at least one secondary before initiating
a new chunk migration.

As in the case before 2.2.1, after all relevant chunks are written
to the destination shard, the balancer waits for replication to
a majority of members (the equivalent of ``w: majority``) before
deleting the chunks from the source shard.

The ``_secondaryThrottle`` behavior of 2.2.1 and later smoothes out
the load somewhat over the duration of the migration, eliminating
the bottleneck. See also :v2.2:`Secondary Throttle in the v2.2 Manual
</tutorial/configure-sharded-cluster-balancer/#sharded-cluster-config-secondary-throttle>`.

While copying and deleting data during migrations, the balancer
waits for :ref:`replication to secondaries
<replica-set-write-concern>` for every document. This slows the potential
speed of a chunk migration but ensures that a large number of chunk
migrations *cannot* affect the availability of a sharded cluster.

See also :v2.2:`Secondary Throttle in the v2.2 Manual
</tutorial/configure-sharded-cluster-balancer/#sharded-cluster-config-secondary-throttle>`.
.. versionchanged:: 2.4
``_secondaryThrottle`` became the default mode for all balancer and
:dbcommand:`moveChunk` operations. The balancer does not require
``_secondaryThrottle`` to be ``true``. To revert to previous
behavior, set ``_secondaryThrottle`` to ``false``.

.. BACKGROUND NOTES
Specifically, secondary throttle affects the first and fourth
phases (informal phases) of chunk migration. Migration can happen during
the second and third phases (the "steady state"):
1) copies the documents in the chunk from shardA to shardB
2) continues to copy over ongoing changes that occurred during the initial copy step,
as well as current changes to that chunk range
3) Stop writes, allow shardB to get final changes, commit migration to config server
4) cleanup now-inactive data on shardA in chunk range (once all cursors are done)

When you :ref:`change the value
<sharded-cluster-config-secondary-throttle>` of ``_secondaryThrottle``,
the balancer may not immediately require the new write concern on
delete and insert operations. To ensure that ``_secondaryThrottle``
takes effect immediately, stop the balancer and restart it with the
selected value of ``_secondaryThrottle``.

53 changes: 9 additions & 44 deletions source/tutorial/configure-sharded-cluster-balancer.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -117,53 +117,18 @@ megabytes.
Require Replication before Chunk Migration (Secondary Throttle)
---------------------------------------------------------------

.. versionadded:: 2.2.1
``_secondaryThrottle`` became an option to the balancer and to
command :dbcommand:`moveChunk`. ``_secondaryThrottle`` makes it
possible to require the balancer wait for replication to
secondaries during migrations.

.. versionchanged:: 2.4
``_secondaryThrottle`` became the default mode for all balancer and
:dbcommand:`moveChunk` operations.

Before 2.2.1, the write operations required to migrate chunks between
shards do not need to replicate to secondaries in order to
succeed. However, you can configure the balancer to require migration
related write operations to replicate to secondaries. This throttles
or slows the migration process and in doing so reduces the potential
impact of migrations on a sharded cluster.

You can throttle migrations by enabling the balancer's
``_secondaryThrottle`` parameter. When enabled, secondary throttle
requires a ``{ w : 2 }`` write concern on delete and insertion
operations, so that every operation propagates to at least one
secondary before the balancer issues the next operation.

Starting with version 2.4 the default secondaryThrottle value is
``true``. To revert to previous behavior, set ``_secondaryThrottle``
to ``false``.

.. BACKGROUND NOTES
Specifically, secondary throttle affects the first and fourth
phases (informal phases) of chunk migration. Migration can happen during
the second and third phases (the "steady state"):
1) copies the documents in the chunk from shardA to shardB
2) continues to copy over ongoing changes that occurred during the initial copy step,
as well as current changes to that chunk range
3) Stop writes, allow shardB to get final changes, commit migration to config server
4) cleanup now-inactive data on shardA in chunk range (once all cursors are done)

You enable or disable ``_secondaryThrottle`` directly in the
:data:`~config.settings` collection in the :ref:`config database
<config-database>` by running the following commands from a
:program:`mongo` shell, connected to a :program:`mongos` instance:
``_secondaryThrottle`` affects the write concern of
the balancer and the :dbcommand:`moveChunk` command. See
:ref:`chunk-migration-write-concern` for a detailed explanation of
write concern with and without ``_secondaryThrottle`` set to ``true``.

Run the following operations from a :program:`mongo` shell that
is connected to a :program:`mongos` instance to enable or disable
``_secondaryThrottle`` directly in the :data:`~config.settings`
collection of the :ref:`config database <config-database>`:

.. code-block:: javascript

use config
db.settings.update( { "_id" : "balancer" } , { $set : { "_secondaryThrottle" : true } } , { upsert : true } )

You also can enable secondary throttle when issuing the
:dbcommand:`moveChunk` command by setting ``_secondaryThrottle`` to
``true``. For more information, see :dbcommand:`moveChunk`.