2006 clean up orphaned and move chunk use w majority for deletion

source/core/sharding-chunk-migration.txt

@@ -98,13 +98,36 @@ Chunk Migration Write Concern
 
 .. versionchanged:: 2.4
 
-   While copying and deleting data during migrations, the balancer
-   waits for :ref:`replication to secondaries
-   <replica-set-write-concern>`. This slows the potential speed of a
-   chunk migration but increases reliability and ensures that a large
-   number of chunk migrations *cannot* affect the availability of a
-   sharded cluster.
-
+   ``secondaryThrottle`` defaults to ``true``. With
+   ``secondaryThrottle`` set to ``true``, the :dbcommand:`moveChunk`
+   command and the MongoDB balancer use an approximate write concern
+   of :ref:`w: 2 <write-concern-replica-acknowledged>` for all chunk
+   copy and delete operations. With ``secondaryThrottle`` set to
+   ``false``, chunk copy and delete operations have an approximate
+   write concern of :ref:`w: 1 <write-concern-acknowledged>`.
+
+   Write concern is identified only approximately here, because the
+   operations are internal to MongoDB, and are handled in a more
+   complex way than normal write operations.
+
+   Regardless of the value of ``secondaryThrottle``, certain parts
+   of chunk migration have a stricter replication policy:
+
+   - MongoDB stops application writes to the source shard before
+     updating the config servers with the chunk locations on
+     the destination shard, and restarts application writes
+     afterwards. During that time some chunk migrations may
+     still need time to complete. Those remaining operations
+     have a write concern of ``w: majority`` and :ref:`j: true
+     <write-concern-replica-journaled>`.
+
+   - After a chunk migration occurs on the primary
+     member of the target shard's replica set, the migration must
+     replicate to a majority of the replica set members before
+     the balancer can migrate more chunks to that shard. This
+     is equivalent to replication having a write concern of ``w:
+     majority` for those operations.
+
    See also :v2.2:`Secondary Throttle in the v2.2 Manual
    </tutorial/configure-sharded-cluster-balancer/#sharded-cluster-config-secondary-throttle>`.
 

source/reference/command/cleanupOrphaned-field.yaml

@@ -1,15 +1,3 @@
-object:
-  name: cleanupOrphaned
-  type: command
-field:
-  optional: false
-  type: field
-name: collection
-type: string
-position: 1
-description: |
-  The complete :term:`namespace` of the :term:`collection` to clean.
----
 object:
   name: cleanupOrphaned
   type: command
@@ -18,7 +6,7 @@ field:
   type: field
 name: startingAtKey
 type: string
-position: 2
+position: 1
 description: |
   The :term:`shard key` value that represents the lower bound of the
   cleanup range. The value ``{ }`` is equivalent to ``MinKey``.
@@ -31,9 +19,11 @@ field:
   type: field
 name: secondaryThrottle
 type: bool
-position: 3
+position: 2
 description: |
-  If ``true``, wait until the deletion of each document replicates
-  to at least one other member of the replica set. This is equivalent
-  to a write concern of ``{ w : 2 }``. Defaults to ``false``.
+  If ``true``, wait until each delete replicates to one other member of
+  the replica set. If ``false``, do not wait for replication. Defaults
+  to ``false``. Regardless of the value of ``secondaryThrottle``,
+  after the final delete, :dbcommand:`cleanupOrphaned` waits for all
+  deletes to replicate to a majority of members before returning.
 ...

source/reference/command/cleanupOrphaned.txt

@@ -11,28 +11,28 @@ Definition
 
    .. versionadded:: 2.5.3
 
-   Incrementally deletes :term:`orphaned data <orphaned document>`,
-   or those documents on a shard that have shard key values that do
-   not fall within the ranges the :term:`chunks <chunk>` owned by
-   that shard.
-
-   Orphaned data sometimes results from aborted migrations due to
-   network latency and power loss.
-
-   Run :dbcommand:`cleanupOrphaned` directly on the :program:`mongod`
-   instance on the primary replica set member of the shard. Do not
-   run :dbcommand:`cleanupOrphaned` from a :program:`mongos` instance.
+   :dbcommand:`cleanupOrphaned` is an admin command that
+   incrementally deletes :term:`orphaned data <orphaned document>`,
+   or those documents on a shard that have shard key values that
+   do not fall within the ranges of the :term:`chunks <chunk>`
+   owned by that shard. By default, after each delete operation
+   :dbcommand:`cleanupOrphaned` does not wait for replication before
+   starting the next delete operation. When ``secondaryThrottle``
+   is ``true``, :dbcommand:`cleanupOrphaned` waits for each delete
+   to replicate to one secondary member before starting the next
+   delete operation. After the final delete operation, Regardless of
+   the value of ``secondaryThrottle``, :dbcommand:`cleanupOrphaned`
+   waits for all deletes to replicate to a majority of members.
 
    :dbcommand:`cleanupOrphaned` has the following prototype form:
 
    .. code-block:: javascript
 
-      {
-         "cleanupOrphaned": 1,
-         "collection": <database>.<collection>,
+      db.runCommand( {
+         cleanupOrphaned: "<database>.<collection>",
          "startingAtKey": <minimumShardKeyValue>,
-         "secondaryThrottle": true
-      }
+         "secondaryThrottle": <boolean>
+      } )
 
    :dbcommand:`cleanupOrphaned` has the following fields:
 
@@ -51,10 +51,10 @@ Behavior
 You do not need to disable the balancer before running
 :dbcommand:`cleanupOrphaned`.
 
-Do not run :dbcommand:`cleanupOrphaned` from :program:`mongos` instances:
-instead, run :dbcommand:`cleanupOrphaned` while connected directly to
-the :program:`mongod` that is currently the primary replica set member
-of the shard.
+Run :dbcommand:`cleanupOrphaned` on the ``admin`` database of the
+:program:`mongod` that is the primary replica set member of the
+shard to be cleaned. Do not run :dbcommand:`cleanupOrphaned` on a
+:program:`mongos` instance.
 
 Output
 ------
@@ -86,11 +86,6 @@ a subset of the following fields:
      hour timeout. In this case, replication does occur, but only
      after :dbcommand:`cleanupOrphaned` returns.
 
-   .. important::
-
-      Do not use the value of :data:`cleanupOrphaned.ok` as an indicator
-      that :dbcommand:`cleanupOrphaned` found or deleted any documents.
-
 .. data:: cleanupOrphaned.startingAtKey
 
    :dbcommand:`cleanupOrphaned` examines documents from a single
@@ -108,16 +103,17 @@ a subset of the following fields:
    the lower bound of the next chunk on the shard. It does not indicate
    that :dbcommand:`cleanupOrphaned` found or deleted any documents.
 
-Interpreting Log Files
-~~~~~~~~~~~~~~~~~~~~~~
+Log Files
+~~~~~~~~~
 
-Read the :program:`mongod` log file to interpret how many orphaned documents
-:dbcommand:`cleanupOrphaned` deleted. Relevant lines resemble the following:
+The :dbcommand:`cleanupOrphaned` command prints the number of deleted
+documents to the :program:`mongod` log. Relevant output lines resemble
+the following:
 
 .. code-block:: none
 
    m30000| 2013-10-31T15:17:28.972-0400 [conn1] Deleter starting delete for: foo.bar from { _id: -35.0 } -> { _id: -10.0 }, with opId: 128
-   m30000| 2013-10-31T15:17:28.972-0400 [conn1] rangeDeleter deleted 0 documents for foo.bar from { _id: -35.0 } -> { _id: -10.0 } { "stoppedAtKey" : { "_id" : -10 }, "ok" : 1 }
+   m30000| 2013-10-31T15:17:28.972-0400 [conn1] rangeDeleter deleted 0 documents for foo.bar from { _id: -35.0 } -> { _id: -10.0 } { "stoppedAtKey": { "_id": -10 }, "ok": 1 }
 
 .. _cleanuporphaned-examples:
 
@@ -135,66 +131,64 @@ Run the following code in the :program:`mongo` shell:
 .. code-block:: javascript
 
    use admin
-   var nextKey = {};
+   var nextKey = { };
    while (
-            nextKey = db.runCommand( { cleanupOrphaned : "test.user",
-                                       startingFromKey : nextKey } ).stoppedAtKey
+            nextKey = db.runCommand( { cleanupOrphaned: "test.user",
+                                       startingFromKey: nextKey } ).stoppedAtKey
          ) { printjson(nextKey); }
 
 Specify a Different Initial Key
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-In the ``testdb`` database, with a sharded ``info`` collection,
-given a shard that has only a single chunk, where :method:`sh.status`
-reports the following range:
+For a sharded collection ``info`` in the ``test`` database, a shard
+owns a single chunk with the following range:
 
 .. code-block:: javascript
+
+   { x: 1 } => { x: 10 }
 
-   { x : 1 } => { x : 10 }
-
-The shard has two ranges not owned by the chunk:
+The shard also contains documents whose shard keys fall in two ranges
+*not* owned by the shard:
 
 .. code-block:: javascript
 
-   { x : MinKey } => { x : 1 }
+   { x: MinKey } => { x: 1 } 
+   { x: 10 } => { x: MaxKey } 
 
-   { x : 10 } => { x : MaxKey }
+From the ``admin`` database, run the :dbcommand:`cleanupOrphaned`
+command to clean up the orphaned documents.
 
-Use one of the following commands to clean up the first range:
+To clean up the range ``{ x: MinKey } => { x: 1 }``, use one of
+the following commands:
 
 .. code-block:: javascript
 
-   {
-      "cleanupOrphaned": 1,
-      "collection": testdb.info,
-      "startingAtKey": { x : 0 },
+   use admin
+   db.runCommand( {
+      "cleanupOrphaned": "test.info",
+      "startingAtKey": { x: 0 },
       "secondaryThrottle": true
-   }
+   } )
 
 .. code-block:: javascript
 
-   {
-      "cleanupOrphaned": 1,
-      "collection": testdb.info,
+   use admin
+   db.runCommand( {
+      "cleanupOrphaned": "test.info",
       "secondaryThrottle": true
-   }
+   } )
 
-To clean the second range, use one of the following commands:
+To clean up the range ``{ x: 10 } => { x: MaxKey }``,  use the following
+command:
 
 .. code-block:: javascript
 
-   {
-      "cleanupOrphaned": 1,
-      "collection": testdb.info,
-      "startingAtKey": { x : 2 },
+   use admin
+   db.runCommand( {
+      "cleanupOrphaned": "test.info",
+      "startingAtKey": { x: 10 },
       "secondaryThrottle": true
-   }
+   } )
 
-.. code-block:: javascript
+.. admin-only
 
-   {
-      "cleanupOrphaned": 1,
-      "collection": testdb.info,
-      "startingAtKey": { x : 10 },
-      "secondaryThrottle": true
-   }