In Review - DOCS-3350 improve backup and restore steps

source/includes/fact-2.0-fsync-lock-change.rst

@@ -0,0 +1,10 @@
+.. versionchanged:: 2.0
+   MongoDB 2.0 added :method:`db.fsyncLock()` and
+   :method:`db.fsyncUnlock()` helpers to the :program:`mongo`
+   shell.  Prior to this version, use the :dbcommand:`fsync`
+   command with the ``lock`` option, as follows:
+
+.. code-block:: javascript
+
+   db.runCommand( { fsync: 1, lock: true } );
+   db.runCommand( { fsync: 1, lock: false } );

source/includes/steps-backup-sharded-cluster-with-snapshots.yaml

@@ -0,0 +1,86 @@
+title: Disable the balancer.
+stepnum: 1
+ref: disable-balancer
+action:
+  pre: |
+    Disable the :term:`balancer` process that equalizes the distribution
+    of data among the :term:`shards <shard>`. To disable the balancer, use
+    the :method:`sh.stopBalancer()` method in the :program:`mongo` shell.
+    For example:
+  language: sh
+  code: |
+    use config
+    sh.stopBalancer()
+post: |
+  For more information, see the
+  :ref:`sharding-balancing-disable-temporarily` procedure.
+---
+title: Lock one secondary member of each replica set in each shard.
+stepnum: 2
+ref: lock
+content: |
+  Lock one secondary member of each replica set in each shard so that your
+  backups reflect the state of your database at the nearest possible
+  approximation of a single moment in time. Lock these :program:`mongod`
+  instances in as short of an interval as possible.
+
+  To lock a secondary, connect through the :program:`mongo` shell to the
+  secondary member's :program:`mongod` instance and issue the
+  :method:`db.fsyncLock()` method.
+---
+title: Back up one of the config servers.
+stepnum: 3
+ref: backup-config-server
+pre: |
+  Backing up a :ref:`config server <sharding-config-server>` backs up the
+  sharded cluster's metadata. You need back up only one config server, as
+  they all hold the same data. Do one of the following to back up one of
+  the config servers:
+action:
+  - heading: "Create a file-system snapshot of the config server."
+    pre: |
+      Do this **only if** the config server has :term:`journaling
+      <journal>` enabled. Use the procedure in
+      :doc:`/tutorial/backup-with-filesystem-snapshots`. **Never** use
+      :method:`db.fsyncLock()` on config databases.
+  - heading: "Create a database dump to backup the config server."
+    pre: |
+      Issue :program:`mongodump` against one of the config
+      :program:`mongod` instances or via the :program:`mongos`. If you are
+      running MongoDB 2.4 or later with the :option:`--configsvr <mongod
+      --configsvr>` option, then include the :option:`--oplog <mongod
+      --oplog>` option to ensure that the dump includes a partial oplog
+      containing operations from the duration of the mongodump operation.
+      For example:
+    language: sh
+    code: |
+      mongodump --oplog --db config
+---
+title: Back up the replica set members of the shards that you locked.
+stepnum: 4
+ref: backup-locked-shards
+content: |
+  You may back up the shards in parallel. For each shard, create a
+  snapshot. Use the procedure in
+  :doc:`/tutorial/backup-with-filesystem-snapshots`.
+---
+title: Unlock locked replica set members.
+stepnum: 5
+ref: unlock
+content: |
+  Unlock all locked replica set members of each shard using the
+  :method:`db.fsyncUnlock()` method in the :program:`mongo` shell.
+---
+title: Enable the balancer.
+stepnum: 6
+ref: enable-balancer
+action:
+  pre: |
+    Re-enable the balancer with the :method:`sh.setBalancerState()`
+    method. Use the following command sequence when connected to the
+    :program:`mongos` with the :program:`mongo` shell:
+  language: javascript
+  code: |
+    use config
+    sh.setBalancerState(true)
+...

source/includes/steps-backup-without-journaling.yaml

@@ -0,0 +1,30 @@
+title: Flush writes to disk and lock the database to prevent further writes.
+stepnum: 1
+ref: flush
+action:
+  pre: |
+    To flush writes to disk and to "lock" the database, issue the
+    :method:`db.fsyncLock()` method in the :program:`mongo` shell:
+  language: javascript
+  code: |
+    db.fsyncLock();
+---
+title: "Perform the backup operation described in :ref:`lvm-backup-operation`."
+stepnum: 2
+ref: backup
+---
+title: "After the snapshot completes, unlock the database."
+stepnum: 3
+ref: unlock
+action:
+  - pre: |
+      To unlock the database after the snapshot has completed, use the
+      following command in the :program:`mongo` shell:
+    language: javascript
+    code: |
+      db.fsyncUnlock();
+post: |
+  .. include:: /includes/warning-fsync-lock-mongodump.rst
+
+  .. include:: /includes/fact-2.0-fsync-lock-change.rst
+...

source/includes/steps-recover-data-with-repairpath.yaml

@@ -0,0 +1,31 @@
+title: "Start :program:`mongod` using the option to replace the original files with the repaired files."
+stepnum: 1
+ref: start-with-repair
+action:
+  pre: |
+    Start the :program:`mongod` instance using the :option:`--repair
+    <mongod --repair>` option **and** the :option:`--repairpath <mongod
+    --repairpath>` option. Issue a command similar to the following:
+  language: sh
+  code: |
+    mongod --dbpath /data/db --repair --repairpath /data/db0
+post: |
+  When this completes, the new repaired data files will be in the
+  ``/data/db0`` directory.
+---
+title: "Start :program:`mongod` with the new data directory."
+stepnum: 2
+ref: start-with-new-path
+action:
+  pre: |
+    Start :program:`mongod` using the following invocation to point the
+    :setting:`~storage.dbPath` at ``/data/db0``:
+  language: sh
+  code: |
+    mongod --dbpath /data/db0
+post: |
+  Once you confirm that the data files are operational you may delete or
+  archive the old data files in the ``/data/db`` directory. You may also
+  wish to move the repaired files to the old database location or update
+  the :setting:`~storage.dbPath` to indicate the new location.
+...

source/includes/steps-recover-data-without-repairpath.yaml

@@ -0,0 +1,39 @@
+title: Remove the stale lock file.
+stepnum: 1
+ref: remove-stale-lock-file
+action:
+  pre: |
+    For example:
+  language: sh
+  code: |
+    rm /data/db/mongod.lock
+post: |
+  Replace ``/data/db`` with your :setting:`~storage.dbPath` where your
+  MongoDB instance's data files reside.
+---
+title: "Start :program:`mongod` using the option to replace the original files with the repaired files."
+stepnum: 2
+ref: start-with-repair
+action:
+  pre: |
+    Start the :program:`mongod` instance using the :option:`--repair
+    <mongod --repair>` option, which replaces the original data files with
+    the repaired data files. Issue a command similar to the following:
+  language: sh
+  code: |
+    mongod --dbpath /data/db --repair
+post: |
+  When this completes, the repaired data files will replace the original
+  data files in the ``/data/db`` directory.
+---
+title: "Start :program:`mongod` as usual."
+stepnum: 3
+ref: start-as-usual
+action:
+  pre: |
+    Start :program:`mongod` using the following invocation to point the
+    :setting:`~storage.dbPath` at ``/data/db``:
+  language: sh
+  code: |
+    mongod --dbpath /data/db
+...

source/includes/steps-restore-primary-from-backup.yaml

@@ -0,0 +1,63 @@
+title: Obtain backup MongoDB Database files.
+stepnum: 1
+ref: obtain-backup-files
+content: |
+  The backup files may come from a :doc:`file system snapshot
+  </tutorial/backup-with-filesystem-snapshots>`. The `MongoDB Management
+  Service (MMS)
+  <https://mms.mongodb.com/?pk_campaign=mongodb-docs-restore-rs-tutorial>`_
+  produces MongoDB database files for :mms:`stored snapshots
+  </backup/tutorial/restore-from-snapshot/>` and :mms:`point and time
+  snapshots </backup/tutorial/restore-from-point-in-time-snapshot/>`. You
+  can also use :program:`mongorestore` to restore database files using
+  data created with :program:`mongodump`. See
+  :doc:`/tutorial/backup-with-mongodump` for more information.
+---
+title: "Start a :program:`mongod` using data files from the backup as the data path."
+stepnum: 2
+ref: start
+action:
+  pre: |
+    The following example uses ``/data/db`` as the data path, as specified
+    in the :setting:`dbpath` setting:
+  language: sh
+  code: |
+    mongod --dbpath /data/db
+---
+title: "Convert the standalone :program:`mongod` to a single-node replica set"
+stepnum: 3
+ref: convert
+action:
+  pre: |
+    Convert the standalone :program:`mongod` process to a single-node
+    replica set by shutting down the :program:`mongod` instance, and
+    restarting it with the :option:`--replSet <mongod --replSet>` option,
+    as in the following example:
+  language: sh
+  code: |
+    mongod --dbpath /data/db --replSet <replName>
+post: |
+  Optionally, you can explicitly set a :setting:`~replication.oplogSizeMB`
+  to control the size of the :term:`oplog` created for this replica set
+  member.
+---
+title: Connect to the :program:`mongod` instance.
+stepnum: 4
+ref: connect
+action:
+  pre: |
+    For example, first issue the following to connect:
+  language: sh
+  code: |
+    mongo
+---
+title: Initiate the new replica set.
+stepnum: 5
+ref: initiate
+action:
+  pre: |
+    Issue :method:`rs.initiate()` to initiate the new replica set. For example:
+  language: javascript
+  code: |
+    rs.initiate()
+...

source/includes/steps-restore-secondary-from-backup-directly.yaml

@@ -0,0 +1,28 @@
+title: "Shut down the :program:`mongod` instance that you restored."
+stepnum: 1
+ref: shutdown
+content: |
+  Use :option:`--shutdown <mongod --shutdown>` or
+  :method:`db.shutdownServer()` to ensure a clean shut down.
+---
+title: "Copy the primary's data directory to each secondary."
+stepnum: 2
+ref: copy-data
+content: |
+  Copy the :term:`primary's <primary>` data directory into the
+  :setting:`~storage.dbPath` of the other members of the replica set. The
+  :setting:`~storage.dbPath` is ``/data/db`` by default.
+---
+title: "Start the :program:`mongod` instance that you restored."
+stepnum: 3
+ref: start
+---
+title: Add the secondaries to the replica set.
+stepnum: 4
+ref: add
+content: |
+  In a :program:`mongo` shell connected to the :term:`primary`, add the
+  :term:`secondaries <secondary>` to the replica set using
+  :method:`rs.add()`. See :doc:`/tutorial/deploy-replica-set` for more
+  information about deploying a replica set.
+...

source/includes/steps-restore-secondary-from-backup-initial-sync.yaml

@@ -0,0 +1,12 @@
+title: Ensure that the data directories on the prospective replica set members are empty.
+stepnum: 1
+ref: delete-data
+---
+title: Add each prospective member to the replica set.
+stepnum: 2
+ref: add-members
+content: |
+  When you add a member to the replica set, :ref:`Initial Sync
+  <replica-set-initial-sync>` copies the data from the :term:`primary` to
+  the new member.
+...

source/includes/steps-restore-sharded-cluster-database-dump.yaml

@@ -0,0 +1,47 @@
+stepnum: 1
+source:
+  file: steps-restore-sharded-cluster-with-snapshots.yaml
+  ref: shutdown-cluster
+---
+title: Restore the data files.
+stepnum: 2
+ref: restore
+pre: |
+  One each server, use :program:`mongorestore` to restore the database
+  dump to the location where the :program:`mongod` instance will access
+  the data.
+action:
+  pre: |
+    The following example restores a database dump located at
+    ``/opt/backup/`` to the ``/data/`` directory. This requires that there
+    are no active :program:`mongod` instances attached to the ``/data``
+    directory.
+  language: sh
+  code: |
+    mongorestore --dbpath /data /opt/backup
+---
+stepnum: 3
+source:
+  file: steps-restore-sharded-cluster-with-snapshots.yaml
+  ref: restart-config-servers
+---
+stepnum: 4
+source:
+  file: steps-restore-sharded-cluster-with-snapshots.yaml
+  ref: shard-hostname
+---
+stepnum: 5
+source:
+  file: steps-restore-sharded-cluster-with-snapshots.yaml
+  ref: restart-shards
+---
+stepnum: 6
+source:
+  file: steps-restore-sharded-cluster-with-snapshots.yaml
+  ref: restart-mongos-instances
+---
+stepnum: 7
+source:
+  file: steps-restore-sharded-cluster-with-snapshots.yaml
+  ref: ensure
+...