Skip to content

DOCS-13539: replSetReconfig allows changing one voting member at a time #4147

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jun 9, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 35 additions & 11 deletions source/includes/extracts-4.4-changes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -421,7 +421,7 @@ content: |
---
ref: 4.4-replSetGetConfiguration-commitmentStatus
content: |
Starting in MongoDB 4.4, the :dbcommand:`replSetGetConfig` command
The :dbcommand:`replSetGetConfig` command
can specify a new option :ref:`commitmentStatus: true
<replSetGetConfig-commitmentStatus>` when run on the primary. When
run with the option, the command includes in the output a
Expand All @@ -446,18 +446,42 @@ content: |
ref: 4.4-replSetReconfig-majority
content: |

Starting in MongoDB 4.4, :dbcommand:`replSetReconfig` waits until a
majority of members install the replica configuration before
returning success. First, the operation waits until the *current*
configuration is committed before installing the *new* configuration
on the primary. The operation then waits until a majority of members
install the *new* configuration before returning successfully.
The :dbcommand:`replSetReconfig` command waits until a majority of
members install the replica configuration before returning success.
First, the operation waits until the *current* configuration is
committed before installing the *new* configuration on the primary.
The operation then waits until a majority of members install the
*new* configuration before returning successfully.
:dbcommand:`replSetReconfig` waits indefinitely by default. MongoDB
4.4 adds the optional :ref:`maxTimeMS
<replSetReconfig-cmd-maxTimeMS>` parameter for specifying the maximum
amount of time to wait for the operation to return successfully. See
:ref:`replSetReconfig-safe-reconfig` for more information on the
reconfiguration process.
<replSetReconfig-cmd-maxTimeMS>` parameter to
:dbcommand:`replSetReconfig` for specifying the maximum amount of
time to wait for the operation to return successfully. See
:ref:`replSetReconfig-cmd-majority-install` for more information.
---
ref: 4.4-replSetReconfig-single-node-voting-change
content: |

The :dbcommand:`replSetReconfig` command allows adding or removing no
more than ``1`` :rsconf:`voting <members[n].votes>` member at a time.
To add or remove multiple voting members, issue a series of
:dbcommand:`replSetReconfig` or :method:`rs.reconfig()` operations to
add or remove one member at a time. See
:ref:`replSetReconfig-cmd-single-node` for more information.
---
ref: 4.4-replSetGetConfig-term-field
content: |

MongoDB 4.4 adds the :rsconf:`term` field to the replica set
:ref:`configuration document <replica-set-configuration-document>`.
Replica set members use :rsconf:`term` and :rsconf:`version` to
achieve consensus on the "newest" replica configuration. Setting
:ref:`featureCompatibilityVersion (fCV) : "4.4" <set-fcv>`
implicitly performs a :dbcommand:`replSetReconfig` to add the
:rsconf:`term` field to the configuration document and blocks until
the new configuration propagates to a majority of replica set
members. Similarly, downgrading to ``fCV : "4.2"`` implicitly
performs a reconfiguration to remove the :rsconf:`term` field.

---
ref: 4.4-changes-migrate-jumbo-chunks
Expand Down
96 changes: 96 additions & 0 deletions source/includes/extracts-replSetReconfig.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
ref: replSetReconfig-majority
content: |

Starting in MongoDB 4.4, |reconfig| waits until a majority of replica
set members install the new replica configuration before returning
success. Replica set members propagate their replica configuration
via :ref:`heartbeats <replica-set-heartbeats>`. Whenever a member
learns of a configuration with a higher :rsconf:`version` and
:rsconf:`term`, it installs the new configuration. The
reconfiguration process has two distinct 'waiting' phases:

\1) Wait for the current configuration to be *committed* before installing the new configuration.
The "current" configuration refers to the replica configuration in
use by the primary at the time |reconfig| is issued.

.. _replSetReconfig-configuration-committed:

A configuration is committed when:

- A majority of replica set members have installed the current
configuration, *and*

- All writes which were majority committed in the previous
configuration have also replicated to a majority in the
current configuration.

Typically, the current configuration has already been installed on
a majority of replica set members. However, the majority committed
writes in the previous configuration may not all be committed
in the current configuration. :rsconf:`Delayed
<members[n].slaveDelay>` members *or* members that are
:method:`lagging behind <rs.printSlaveReplicationInfo()>` the
primary can increase the time spent in this phase.

If the operation was issued with a
|reconfig-maxTimeMS| limit *and*
the operation exceeds the limit while waiting, the operation
returns an error and discard the new configuration. The
limit is *cumulative* and does not reset after proceeding to
the next phase.

\2) Wait for a majority of members in the new configuration to install the new configuration.
The "new" configuration refers to the replica configuration
specified to |reconfig|.

The primary installs and begins using the new replica
configuration before propagating the configuration to the
remaining replica set members. The operation only waits for
a majority of members to install the new configuration, and does
not require waiting for the new configuration to be
:ref:`committed <replSetReconfig-configuration-committed>`.

If the operation was issued with a |reconfig-maxTimeMS| limit
*and* the operation exceeds the limit while waiting, the operation
returns an error *but* continues using and propagating the new
configuration.

Issuing a |reconfig-force| reconfiguration
immediately installs the new configuration regardless of the
commitment status of the previous configuration. Force
reconfiguration can cause unexpected behavior, such as the
rollback of :writeconcern:`"majority"` committed write operations.

To check the commitment status of the current replica configuration,
issue :dbcommand:`replSetGetConfig` with the
:ref:`commitmentStatus <replSetGetConfig-commitmentStatus>` parameter
on the replica set :term:`primary`.

---
ref: replSetReconfig-single-node
content: |

Starting in MongoDB 4.4, |reconfig| by default allows adding or
removing no more than ``1`` :rsconf:`voting <members[n].votes>`
member at a time. For example, a new configuration can make at most
*one* of the following changes to the cluster :rsconf:`membership
<members[n].votes>`:

- Adding a new voting replica set member.

- Removing an existing voting replica set member.

- Modifying the :rsconf:`~members[n].votes` for an existing replica
set member.

To add or remove multiple voting members, issue a series of
|reconfig| operations to add or remove one member at a
time.

Issuing a |reconfig-force| reconfiguration
immediately installs the new configuration even if it adds or removes
multiple voting members. Force reconfiguration can cause unexpected
behavior, such as the rollback of :writeconcern:`"majority"`
committed write operations.

...
3 changes: 2 additions & 1 deletion source/includes/fact-rs-non-voting-priority-restriction.rst
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
Non-voting members must have :rsconf:`~members[n].priority` of 0.
Non-voting (i.e. :rsconf:`~members[n].votes` is ``0``) members must have
:rsconf:`~members[n].priority` of 0.
7 changes: 7 additions & 0 deletions source/includes/steps-4.4-upgrade-replica-set.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,13 @@ action:
code: |
db.adminCommand( { setFeatureCompatibilityVersion: "4.4" } )
post: |

Setting :ref:`featureCompatibilityVersion (fCV) : "4.4" <set-fcv>`
implicitly performs a :dbcommand:`replSetReconfig` to add the
:rsconf:`term` field to the configuration document and blocks
until the new configuration propagates to a majority of replica
set members.

This command must perform writes to an internal system
collection. If for any reason the command does not complete
successfully, you can safely retry the command on the primary as
Expand Down
6 changes: 6 additions & 0 deletions source/includes/steps-4.4-upgrade-sharded-cluster.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,12 @@ content: |

db.adminCommand( { setFeatureCompatibilityVersion: "4.4" } )

Setting :ref:`featureCompatibilityVersion (fCV) : "4.4" <set-fcv>`
implicitly performs a :dbcommand:`replSetReconfig` on each shard to
add the :rsconf:`term` field to the shard replica configuration
document and blocks until the new configuration propagates to a
majority of replica set members.

This command must perform writes to an internal system collection.
If for any reason the command does not complete successfully, you
can safely retry the command on the :binary:`~bin.mongos` as the
Expand Down
88 changes: 21 additions & 67 deletions source/reference/command/replSetReconfig.txt
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ replSetReconfig
replica configuration to propagate to a majority of replica
set members. Setting ``maxTimeMS`` may result
in the operation failing *before* it can apply the new
configuration. See :ref:`replSetReconfig-safe-reconfig`
configuration. See :ref:`replSetReconfig-cmd-majority-install`
for more information.

.. versionadded:: 4.4
Expand All @@ -72,78 +72,32 @@ replSetReconfig
Behavior
--------

``term`` Replica Configuration Field
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

MongoDB 4.4 adds the :rsconf:`term` field to the replica set
configuration document. The :rsconf:`term` field is set by the
:term:`primary` replica set member. The primary ignores the
:rsconf:`term` field if set explicitly in the
:dbcommand:`replSetReconfig` operation.

.. _replSetReconfig-safe-reconfig:

Safe Replica Set Reconfiguration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Starting in MongoDB 4.4, :dbcommand:`replSetReconfig` waits until a
majority of replica set members install the new replica configuration
before returning success. Replica set members propagate their replica
configuration via :ref:`heartbeats <replica-set-heartbeats>`. Whenever a
member learns of a configuration with a higher :rsconf:`version` and
:rsconf:`term`, it accepts and installs the new configuration. The
reconfiguration process has two distinct 'waiting' phases:

\1) Wait for the current configuration to be *committed* before installing the new configuration.
The "current" configuration refers to the replica configuration in use
by the primary at the time :dbcommand:`replSetReconfig` is issued.

The current configuration is *committed* when:

- A majority of replica set members have installed the current
configuration, *and*

- All writes which were majority committed in the previous
configuration have also replicated to a majority in the
current configuration.

The current configuration has typically already been installed on a
majority of replica set members and does not require additional
waiting. The operation *may* wait a significant amount of time for a
majority of replica members to commit specific oplog entries.
:rsconf:`Delayed <members[n].slaveDelay>` members *or* members that
are :method:`lagging behind <rs.printSlaveReplicationInfo()>`
the primary can increase the time spent in this phase.

If the operation was issued with a
:ref:`maxTimeMS <replSetReconfig-cmd-maxTimeMS>` limit *and*
the operation exceeds the limit while waiting,
return an error and discard the new configuration. The
limit is *cumulative* and does not reset after proceeding to
the next phase.

\2) Wait for a majority of members in the new configuration to install the new configuration.
The "new" configuration refers to the replica configuration
specified to :dbcommand:`replSetReconfig`.

The primary installs and begins using the new replica
configuration before propagating the configuration to the
remaining replica set members. The operation only waits for
a majority of members to install the new configuration, and does
not require waiting for the new configuration to be *committed*.

If the operation was issued with a :ref:`maxTimeMS
<replSetReconfig-cmd-maxTimeMS>` limit *and* the operation
exceeds the limit while waiting, return an error *but* continue
using and propagating the new configuration.

Issuing a :ref:`force <replSetReconfig-cmd-force>` reconfiguration
immediately installs the new configuration regardless of the commitment
status of the previous configuration. Force reconfiguration can cause
unexpected behavior, such as the rollback of :writeconcern:`"majority"`
committed write operations.

To check the commitment status of the current replica configuration,
issue :dbcommand:`replSetGetConfig` with the
:ref:`commitmentStatus <replSetGetConfig-commitmentStatus>` parameter
on the replica set :term:`primary`. (*New in 4.4*)
.. |reconfig| replace:: :dbcommand:`replSetReconfig`
.. |reconfig-force| replace:: :ref:`force <replSetReconfig-cmd-force>`
.. |reconfig-maxTimeMS| replace:: :ref:`maxTimeMS <replSetReconfig-cmd-maxTimeMS>`

.. _replSetReconfig-cmd-single-node:

Reconfiguration Can Add or Remove No More than One Voting Member at a Time
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: /includes/extracts/replSetReconfig-single-node.rst

.. _replSetReconfig-cmd-majority-install:

Reconfiguration Waits Until a Majority of Members Install the Replica Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: /includes/extracts/replSetReconfig-majority.rst

Access Control
~~~~~~~~~~~~~~
Expand Down
86 changes: 20 additions & 66 deletions source/reference/method/rs.reconfig.txt
Original file line number Diff line number Diff line change
Expand Up @@ -96,78 +96,32 @@ Definition
Behavior
--------

``term`` Replica Configuration Field
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

MongoDB 4.4 adds the :rsconf:`term` field to the replica set
configuration document. The :rsconf:`term` field is set by the
:term:`primary` replica set member. The primary ignores the
:rsconf:`term` field if set explicitly in the
:method:`rs.reconfig()` operation.

.. _rs-reconfig-safe-reconfig:

Safe Replica Set Reconfiguration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Starting in MongoDB 4.4, :method:`rs.reconfig()` waits until a
majority of replica set members install the new replica configuration
before returning success. Replica set members propagate their replica
configuration via :ref:`heartbeats <replica-set-heartbeats>`. Whenever a
member learns of a configuration with a higher :rsconf:`version` and
:rsconf:`term`, it accepts and installs the new configuration. The
reconfiguration process has two distinct 'waiting' phases:

\1) Wait for the current configuration to be *committed* before installing the new configuration.
The "current" configuration refers to the replica configuration in use
by the primary at the time :method:`rs.reconfig()` is issued.

The current configuration is committed when:

- A majority of replica set members have installed the current
configuration, *and*

- All writes which were majority committed in the previous
configuration have also replicated to a majority in the
current configuration.

The current configuration has typically already been installed on a
majority of replica set members and does not require additional
waiting. The operation *may* wait a significant amount of time for a
majority of replica members to commit specific oplog entries.
:rsconf:`Delayed <members[n].slaveDelay>` members *or* members that
are :method:`lagging behind <rs.printSlaveReplicationInfo()>`
the primary can increase the time spent in this phase.

If the operation was issued with a
:ref:`maxTimeMS <rs-reconfig-method-maxTimeMS>` limit *and*
the operation exceeds the limit while waiting,
return an error and discard the new configuration. The
limit is *cumulative* and does not reset after proceeding to
the next phase.

\2) Wait for a majority of members in the new configuration to install the new configuration.
The "new" configuration refers to the replica configuration
specified to :method:`rs.reconfig()`.

The primary installs and begins using the new replica
configuration before propagating the configuration to the
remaining replica set members. The operation only waits for
a majority of members to install the new configuration, and does
not require waiting for the new configuration to be *committed*.

If the operation was issued with a :ref:`maxTimeMS
<rs-reconfig-method-maxTimeMS>` limit *and* the operation
exceeds the limit while waiting, return an error *but* continue
using and propagating the new configuration.

Issuing a :ref:`force <rs-reconfig-method-force>` reconfiguration
immediately installs the new configuration regardless of the commitment
status of the previous configuration. Force reconfiguration can cause
unexpected behavior, such as the rollback of :writeconcern:`"majority"`
committed write operations.

To check the commitment status of the current replica configuration,
issue :dbcommand:`replSetGetConfig` with the
:ref:`commitmentStatus <replSetGetConfig-commitmentStatus>` parameter
on the replica set :term:`primary`. (*New in 4.4*)
.. |reconfig| replace:: :method:`rs.reconfig()`
.. |reconfig-force| replace:: :ref:`force <rs-reconfig-method-force>`
.. |reconfig-maxTimeMS| replace:: :ref:`maxTimeMS <rs-reconfig-method-maxTimeMS>`

.. _rs-reconfig-single-node:

Reconfiguration Can Add or Remove No More than One Voting Member at a Time
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: /includes/extracts/replSetReconfig-single-node.rst

.. _rs-reconfig-majority-install:

Reconfiguration Waits Until a Majority of Members Install the Replica Configuration
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. include:: /includes/extracts/replSetReconfig-majority.rst

Access Control
~~~~~~~~~~~~~~
Expand Down
Loading