DOCS-15340 Balancer Policy redux (#1990) (#2278)

* DOCS-15340 redux

* updates

* prep

* build errors

* tech feedback

* tech review feedback

* restores original SVG

* tech feedback

* internal review updates

Author: jmd-mongo

source/core/hashed-sharding.txt

@@ -130,13 +130,11 @@ Shard a Populated Collection
 
 If you shard a populated collection using a hashed shard key:
 
-- The sharding operation creates the initial chunk(s) to cover the
-  entire range of the shard key values. The number of chunks created
-  depends on the :ref:`configured chunk size <sharding-chunk-size>`.
+- The sharding operation creates an initial chunk to cover all of the
+  shard key values.
 
-- After the initial chunk creation, the balancer migrates these initial
-  chunks across the shards as appropriate as well as manages the chunk
-  distribution going forward.
+- After the initial chunk creation, the balancer moves ranges of the
+  initial chunk when it needs to balance data.
 
 Shard an Empty Collection
 ~~~~~~~~~~~~~~~~~~~~~~~~~

source/core/ranged-sharding.txt

@@ -64,15 +64,9 @@ to use as the :term:`shard key`.
 Shard a Populated Collection
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-If you shard a populated collection:
-
-- The sharding operation creates the initial chunk(s) to cover the
-  entire range of the shard key values. The number of chunks created
-  depends on the :ref:`configured chunk size <sharding-chunk-size>`.
-
-- After the initial chunk creation, the balancer migrates these initial
-  chunks across the shards as appropriate as well as manages the chunk
-  distribution going forward.
+If you shard a populated collection, only one chunk is created 
+initially. The balancer then migrates ranges from that chunk if 
+necessary according to the configured range size.
 
 Shard an Empty Collection
 ~~~~~~~~~~~~~~~~~~~~~~~~~

source/core/sharded-cluster-config-servers.txt

@@ -27,9 +27,8 @@ the ranges that define the chunks.
 The :binary:`~bin.mongos` instances cache this data and use it to route
 read and write operations to the correct shards. :binary:`~bin.mongos`
 updates the cache when there are metadata changes for the cluster, such
-as :ref:`sharding-chunk-splits` or :doc:`adding a
-shard</tutorial/add-shards-to-shard-cluster>`. Shards also read chunk
-metadata from the config servers.
+as :ref:`adding a shard <sharding-procedure-add-shard>`. Shards also read 
+chunk metadata from the config servers.
 
 The config servers also store :doc:`authentication` configuration
 information such as :doc:`Role-Based Access

source/core/sharding-balancer-administration.txt

@@ -13,15 +13,17 @@ Sharded Cluster Balancer
    :depth: 1
    :class: singlecol
 
-The MongoDB balancer is a background process that monitors the number of
-:term:`chunks <chunk>` on each :term:`shard`. When the number of chunks on a
-given shard reaches specific :ref:`migration thresholds
-<sharding-migration-thresholds>`, the balancer attempts to automatically
-migrate chunks between shards and reach an equal number of chunks per shard.
-
-The balancing procedure for :term:`sharded clusters <sharded cluster>` is
-entirely transparent to the user and application layer, though there may be
-some performance impact while the procedure takes place.
+The MongoDB balancer is a background process that monitors the amount of
+data on each :term:`shard` for each sharded collection. When the amount 
+of data for a sharded collection on a given shard reaches specific 
+:ref:`migration thresholds <sharding-migration-thresholds>`, the balancer 
+attempts to automatically migrate data between shards and reach an even 
+amount of data per shard while respecting the :ref:`zones 
+<zone-sharding>`. By default, the balancer process is always enabled. 
+
+The balancing procedure for :term:`sharded clusters <sharded cluster>` 
+is entirely transparent to the user and application layer, though there 
+may be some performance impact while the procedure takes place.
 
 .. include:: /images/sharding-migrating.rst
 
@@ -39,43 +41,28 @@ The balancer runs on the primary of the config server replica set
 .. _sharding-balancing-internals:
 .. _sharding-internals-balancing:
 
-Cluster Balancer
-----------------
-
-The :term:`balancer` process is responsible for redistributing the
-chunks of a sharded collection evenly among the shards for every
-sharded collection. By default, the balancer process is always enabled.
-
-To address uneven chunk distribution for a sharded collection, the
-balancer :doc:`migrates chunks </core/sharding-balancer-administration>` from
-shards with more chunks to shards with a fewer number of chunks. The
-balancer migrates the chunks until there is an even
-distribution of chunks for the collection across the shards. For details
-about chunk migration, see :ref:`chunk-migration-procedure`.
+Balancer Internals
+------------------
 
-.. include:: /includes/fact-archiveMovedChunks.rst
-
-Chunk migrations carry some overhead in terms of bandwidth and
-workload, both of which can impact database performance. [#auto-distribute]_ The
-:term:`balancer` attempts to minimize the impact by:
+Range migrations carry some overhead in terms of bandwidth and
+workload, both of which can impact database performance. 
+The :term:`balancer` attempts to minimize the impact by:
 
 - Restricting a shard to at most one migration at any given time.
-  Specifically, a shard cannot participate in multiple chunk migrations
-  at the same time. To migrate multiple chunks from a shard, the
-  balancer migrates the chunks one at a time.
+  Specifically, a shard cannot participate in multiple data migrations
+  at the same time. The balancer migrates ranges one at a time.
 
-  MongoDB can perform parallel chunk migrations, but a shard can
+  MongoDB can perform parallel data migrations, but a shard can
   participate in at most one migration at a time. For a sharded cluster
   with *n* shards, MongoDB can perform at most *n/2* (rounded down)
-  simultaneous chunk migrations.
+  simultaneous migrations.
 
-  See also :ref:`chunk-migration-queuing`.
+  See also :ref:`range-migration-queuing`.
 
-- Starting a balancing round **only** when the difference in the
-  number of chunks between the shard with the greatest number of chunks
-  for a sharded collection and the shard with the lowest number of
-  chunks for that collection reaches the :ref:`migration threshold
-  <sharding-migration-thresholds>`.
+- Starting a balancing round **only** when the difference in the amount 
+  of data between the shard with the most data for a sharded collection 
+  and the shard with the least data for that collection reaches the 
+  :ref:`migration threshold <sharding-migration-thresholds>`.
 
 You may disable the balancer temporarily for maintenance. See
 :ref:`sharding-balancing-disable-temporally` for details.
@@ -91,76 +78,69 @@ Window <sharding-schedule-balancing-window>` for details.
 
 .. seealso::
 
-   :doc:`/tutorial/manage-sharded-cluster-balancer`
-
-
-.. [#auto-distribute]
-
-   .. include:: /includes/extracts/zoned-sharding-shard-operation-chunk-distribution.rst
-
-   .. include:: /includes/extracts/zoned-sharding-shard-operation-chunk-distribution-hashed-short.rst
-
-   See :ref:`pre-define-zone-range-hashed-example` for an example.
+   :ref:`<sharded-cluster-balancer>`
 
 
 Adding and Removing Shards from the Cluster
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 Adding a shard to a cluster creates an imbalance, since the new
-shard has no chunks. While MongoDB begins migrating data to the new
-shard immediately, it can take some time before the cluster balances. See the
-:doc:`/tutorial/add-shards-to-shard-cluster` tutorial for instructions on
-adding a shard to a cluster.
+shard has no data. While MongoDB begins migrating data to the new
+shard immediately, it can take some time before the cluster balances. 
+See the :ref:`Add Shards to a Cluster <sharding-procedure-add-shard>` 
+tutorial for instructions on adding a shard to a cluster.
 
-Removing a shard from a cluster creates a similar imbalance, since chunks
-residing on that shard must be redistributed throughout the cluster. While
-MongoDB begins draining a removed shard immediately, it can take some time
-before the cluster balances. *Do not* shutdown the servers associated
-to the removed shard during this process. 
+Removing a shard from a cluster creates a similar imbalance, since data
+residing on that shard must be redistributed throughout the cluster. 
+While MongoDB begins draining a removed shard immediately, it can take 
+some time before the cluster balances. *Do not* shutdown the servers 
+associated to the removed shard during this process. 
 
 .. include:: /includes/fact-remove-shard-balance-order.rst
 
-See the :doc:`/tutorial/remove-shards-from-cluster` tutorial for
-instructions on safely removing a shard from a cluster.
+See the :ref:`Remove Shards from a Cluster 
+<remove-shards-from-cluster-tutorial>` tutorial for instructions on 
+safely removing a shard from a cluster.
 
 .. seealso::
 
    :method:`sh.balancerCollectionStatus()`
 
 
 .. _chunk-migration-procedure:
+.. _range-migration-procedure:
 
-Chunk Migration Procedure
+Range Migration Procedure
 -------------------------
 
-All chunk migrations use the following procedure:
+All range migrations use the following procedure:
 
-#. The balancer process sends the :dbcommand:`moveChunk` command to
+#. The balancer process sends the :dbcommand:`moveRange` command to
    the source shard.
 
 #. The source starts the move when it receives an internal 
    :dbcommand:`moveRange` command. During the migration process, 
-   operations to the chunk are sent to the source shard. The source 
-   shard is responsible for incoming write operations for the chunk.
+   operations to the range are sent to the source shard. The source 
+   shard is responsible for incoming write operations for the range.
 
 #. The destination shard builds any indexes required by the source
    that do not exist on the destination.
 
-#. The destination shard begins requesting documents in the chunk and
+#. The destination shard begins requesting documents in the range and
    starts receiving copies of the data. See also
-   :ref:`chunk-migration-replication`.
+   :ref:`range-migration-replication`.
 
-#. After receiving the final document in the chunk, the
+#. After receiving the final document in the range, the
    destination shard starts a synchronization process to ensure that it
    has the changes to the migrated documents that occurred during the
    migration.
 
 #. When fully synchronized, the source shard connects to the
    :term:`config database` and updates the cluster metadata with the new
-   location for the chunk.
+   location for the range.
 
 #. After the source shard completes the update of the metadata,
-   and once there are no open cursors on the chunk, the source shard
+   and once there are no open cursors on the range, the source shard
    deletes its copy of the documents.
 
    .. note::
@@ -174,9 +154,6 @@ All chunk migrations use the following procedure:
 
       :ref:`moveChunk-directory`
 
-The migration process ensures consistency and maximizes the availability of
-chunks during balancing.
-
 .. seealso::
 
    :serverstatus:`shardingStatistics.countDonorMoveChunkLockTimeout`
@@ -189,87 +166,69 @@ Migration Thresholds
 
 To minimize the impact of balancing on the cluster, the
 :term:`balancer` only begins balancing after the distribution of
-chunks for a sharded collection has reached certain thresholds. The
-thresholds apply to the difference in number of :term:`chunks <chunk>`
-between the shard with the most chunks for the collection and the shard
-with the fewest chunks for that collection. The balancer has the
-following thresholds:
-
-.. list-table::
-   :header-rows: 1
-
-   * - Number of Chunks
-     - Migration Threshold
-
-   * - Fewer than 20
-     - 2
+date for a sharded collection has reached certain thresholds. 
 
-   * - 20-79
-     - 4
-
-   * - 80 and greater
-     - 8
-
-The balancer stops running on the target collection when the difference
-between the number of chunks on any two shards for that collection is *less
-than two*, or a chunk migration fails.
+A collection is considered balanced if the difference in data between 
+shards (for that collection) is less than :ref:`migration thresholds 
+<sharding-migration-thresholds>`.
 
 .. seealso::
 
    :method:`sh.balancerCollectionStatus()`
 
 
 .. _chunk-migration-queuing:
+.. _range-migration-queuing:
 .. _asynchronous-chunk-migration-cleanup:
 
-Asynchronous Chunk Migration Cleanup
+Asynchronous Range Migration Cleanup
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-To migrate multiple chunks from a shard, the balancer migrates the
-chunks one at a time. However, the balancer does not wait for the
-current migration's delete phase to complete before starting the next
-chunk migration. See :ref:`sharding-chunk-migration` for the chunk
+To migrate data from a shard, the balancer migrates the
+data one range at a time. However, the balancer does not wait for the 
+current migration's delete phase to complete before starting the next 
+range migration. See :ref:`sharding-range-migration` for the range
 migration process and the delete phase.
 
-This queuing behavior allows shards to unload chunks more quickly in
+This queuing behavior allows shards to unload data more quickly in
 cases of heavily imbalanced cluster, such as when performing initial
 data loads without pre-splitting and when adding new shards.
 
 This behavior also affects the :dbcommand:`moveRange` command, and
 migration scripts that use the :dbcommand:`moveRange` command may
 proceed more quickly.
 
-In some cases, the delete phases may persist longer. Starting in MongoDB
-4.4, chunk migrations are enhanced to be more resilient in the event of
-a failover during the delete phase. Orphaned documents are cleaned up
-even if a replica set's primary crashes or restarts during this phase.
+In some cases, the delete phases may persist longer. Range migrations 
+are enhanced to be more resilient in the event of a failover during the 
+delete phase. Orphaned documents are cleaned up even if a replica set's 
+primary crashes or restarts during this phase.
 
-The ``_waitForDelete``, available as a setting for the balancer as well
-as the :dbcommand:`moveRange` command, can alter the behavior so that
+The ``_waitForDelete`` balancer setting can alter the behavior so that
 the delete phase of the current migration blocks the start of the next
 chunk migration. The ``_waitForDelete`` is generally for internal
-testing purposes. For more information, see
+testing purposes. For more information, see 
 :ref:`wait-for-delete-setting`.
 
 .. _chunk-migration-replication:
+.. _range-migration-replication:
 
-Chunk Migration and Replication
+Range Migration and Replication
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-During chunk migration, the ``_secondaryThrottle`` value determines
-when the migration proceeds with next document in the chunk.
+During range migration, the ``_secondaryThrottle`` value determines
+when the migration proceeds with next document in the range.
 
 In the :data:`config.settings` collection:
 
 - If the ``_secondaryThrottle`` setting for the balancer is set to a
-  write concern, each document move during chunk migration must receive
+  write concern, each document move during range migration must receive
   the requested acknowledgement before proceeding with the next
   document.
 
 - If the ``_secondaryThrottle`` setting for the balancer is set to
-  ``true``, each document move during chunk migration must receive
+  ``true``, each document move during range migration must receive
   acknowledgement from at least one secondary before the migration
-  proceeds with the next document in the chunk. This is equivalent to a
+  proceeds with the next document in the range. This is equivalent to a
   write concern of :writeconcern:`{ w: 2 } <\<number\>>`.
 
 - If the ``_secondaryThrottle`` setting is unset, the migration process
@@ -280,17 +239,16 @@ To update the ``_secondaryThrottle`` parameter for the balancer, see
 :ref:`sharded-cluster-config-secondary-throttle` for an example.
 
 Independent of any ``_secondaryThrottle`` setting, certain phases of
-the chunk migration have the following replication policy:
-
-- MongoDB briefly pauses all application reads and writes to the
-  collection being migrated, on the source shard, before updating the
-  config servers with the new location for the chunk, and resumes the
-  application reads and writes after the update. The chunk move requires
-  all writes to be acknowledged by majority of the members of the
-  replica set both before and after committing the chunk move to config
-  servers.
- 
-- When an outgoing chunk migration finishes and cleanup occurs, all
+the range migration have the following replication policy:
+
+- MongoDB briefly pauses all application reads and writes to the 
+  collection being migrated to on the source shard before updating the
+  config servers with the range location. MongoDB resumes application
+  reads and writes after the update. The range move requires all writes 
+  to be acknowledged by majority of the members of the replica set both 
+  before and after committing the range move to config servers.
+
+- When an outgoing migration finishes and cleanup occurs, all
   writes must be replicated to a majority of servers before further
   cleanup (from other outgoing migrations) or new incoming migrations
   can proceed.
@@ -300,11 +258,12 @@ To update the ``_secondaryThrottle`` setting in the
 :ref:`sharded-cluster-config-secondary-throttle` for an example.
 
 .. _migration-chunk-size-limit:
+.. _migration-range-size-limit:
 
-Maximum Number of Documents Per Chunk to Migrate
+Maximum Number of Documents Per Range to Migrate
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-.. include:: /includes/limits-sharding-maximum-documents-chunk.rst
+.. include:: /includes/limits-sharding-maximum-documents-range.rst
 
 .. _range-deletion-performance-tuning: