Skip to content

DOCP-980 correct oplogReplay behaviour for 3.4+ #3244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 27, 2018
Merged

DOCP-980 correct oplogReplay behaviour for 3.4+ #3244

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2b1c760
DOCSP-980: clarify oplogReplay behaviour for 3.4
schmalliso Jan 4, 2018
2924651
correction to address TOOLS-1035 changes
schmalliso Feb 26, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 51 additions & 18 deletions source/includes/options-mongorestore.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -234,27 +234,56 @@ directive: option
args: null
description: |
After restoring the database dump, replays the :term:`oplog` entries
from the :file:`oplog.bson` file located in the top level of the dump
directory. When used in conjunction with :option:`mongodump --oplog`,
:binary:`~bin.mongorestore --oplogReplay <mongorestore>`
from a bson file.
When used in conjunction with :option:`mongodump --oplog
<--oplog>`, :binary:`~bin.mongorestore --oplogReplay <mongorestore>`
restores the database to the point-in-time backup captured with the
``mongodump --oplog`` command.

``mongodump --oplog`` command.

{{program}} searches for any valid source for the bson file
in the following locations:

- The top level of the dump directory, as in the case of a dump created
with :option:`mongodump --oplog <--oplog>`.

- The path specified by :option:`--oplogFile`.

- ``<dump-directory>/local/oplog.rs.bson``, as in the case of a dump
of the ``oplog.rs`` collection in the ``local`` database
on a :binary:`~bin.mongod` that is a member of a replica set.

- ``<dump-directory>/local/oplog.$main.bson``, as in the case of a dump
of the ``oplog.rs`` collection in the ``local`` database
on a :binary:`~bin.mongod` that uses master-slave replication.

If there is an ``oplog.bson`` file at the top level of the dump
directory **and** a path specified by :option:`--oplogFile`,
{{program}} returns an error.

If there is an ``oplog.bson`` file at the top level of the dump directory,
{{program}} restores that file as the oplog. If there are also bson
files in the ``dump/local`` directory, {{program}} restores them like
normal collections.

If you specify an oplog file using :option:`--oplogFile`,
{{program}} restores that file as the oplog. If there are also bson
files in the ``dump/local`` directory, {{program}} restores them like
normal collections.

If there is both an ``oplog.rs.bson`` and an ``oplog.$main.bson``
file in the ``dump/local`` directory **and** there is no
``oplog.bson`` file at the top level of the dump directory or file
specified with :option:`--oplogFile`, {{program}} returns an error.

For an example of {{role}}, see :ref:`backup-restore-oplogreplay`.

:binary:`~bin.mongorestore --oplogReplay <mongorestore>` replays any
valid :file:`oplog.bson` file found in the top level of the dump
directory. That is, if you have a bson file that contains valid oplog
entries, you can name the file ``oplog.bson`` and move it to the top
level of the dump directory for :binary:`~bin.mongorestore --oplogReplay
<mongorestore>` to replay.

.. note::

When using :binary:`~bin.mongorestore` with {{role}}, you must
restore a full dump of a :term:`replica set` member created
using :binary:`~bin.mongodump --oplog <mongodump>`.
:binary:`~bin.mongorestore` with {{role}} fails if you use any of
When using :binary:`~bin.mongorestore` with {{role}} to restore
a replica set, you must
restore a full dump of a :term:`replica set` member created
using :binary:`~bin.mongodump --oplog <mongodump>`.
:binary:`~bin.mongorestore` with {{role}} fails if you use any of
the following options to limit the data be restored:

- :option:`--db`
Expand Down Expand Up @@ -418,7 +447,7 @@ description: |
.. versionadded:: 3.2

Restores from compressed files or data stream created by
:binary:`mongodump --archive <bin.mongodump>`
:binary:`~bin.mongodump --archive <mongodump>`

To restore from a dump directory that contains compressed files, run
:binary:`~bin.mongorestore` with the new ``--gzip`` option.
Expand Down Expand Up @@ -508,7 +537,11 @@ pre: |

description: |
Specifies the path to the oplog file containing oplog data for the
restore.
restore. Use with :option:`--oplogReplay`.

If you specify {{role}} and there is an :file:`oplog.bson`
file at the top level of the dump directory, {{program}} returns an
error.
optional: true
---
program: mongorestore
Expand Down
91 changes: 46 additions & 45 deletions source/reference/program/mongorestore.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,17 +67,11 @@ Version Compatibility

.. include:: /includes/warning-mongodump-compatibility-2.2.rst

.. COMMENT upon completion of TOOLS-1035, either delete the whole section or delete everything after the semicolon.

Exclude ``system.profile`` Collection
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:binary:`~bin.mongorestore` does not restore the :data:`system.profile
<<database>.system.profile>` collection data; however, if the backup
data includes :data:`system.profile <<database>.system.profile>`
collection data and the collection does not exist in the target
database, :binary:`~bin.mongorestore` creates the collection but does not
insert any data into the collection.
<<database>.system.profile>` collection data.

.. _mongorestore-required-access:

Expand Down Expand Up @@ -213,13 +207,14 @@ running on the localhost interface on port ``27017``.

.. versionadded:: 3.4

You may alternatively use :option:`--nsInclude <mongorestore --nsInclude>` to specify the
canonical name of the collection that you wish to restore rather than
:option:`--collection <mongorestore --collection>`. :option:`--nsInclude <mongorestore --nsInclude>` enables you to specify
the :term:`namespace` of one or more collections that you
wish to include in the restore operation. The following example
restores the ``people`` collection from the ``accounts`` database
in the ``dump/`` sub-directory of the current directory:
You may alternatively use :option:`--nsInclude <mongorestore
--nsInclude>` to specify the canonical name of the collection that you
wish to restore rather than :option:`--collection <mongorestore
--collection>`. :option:`--nsInclude <mongorestore --nsInclude>`
enables you to specify the :term:`namespace` of one or more collections
that you wish to include in the restore operation. The following
example restores the ``people`` collection from the ``accounts``
database in the ``dump/`` sub-directory of the current directory:

.. code-block:: sh

Expand All @@ -232,18 +227,20 @@ Restore Collections Using Wild Cards

.. versionadded:: 3.4

:option:`--nsInclude <mongorestore --nsInclude>` and :option:`--nsExclude <mongorestore --nsExclude>` support specifying
the :term:`namespaces <namespace>` you wish to include or exclude from a restore operation
using asterisks as *wild cards*.
:option:`--nsInclude <mongorestore --nsInclude>` and
:option:`--nsExclude <mongorestore --nsExclude>` support specifying the
:term:`namespaces <namespace>` you wish to include or exclude from a
restore operation using asterisks as *wild cards*.

The following example restores the documents in the ``dump/``
sub-directory of the current directory that match the specified
namespace pattern. The :option:`--nsInclude <mongorestore --nsInclude>` statement specifies to
only restore documents in the ``transactions`` database while
:option:`--nsExclude <mongorestore --nsExclude>` instructs :binary:`~bin.mongorestore` to exclude
collections whose names end with ``_dev``. :binary:`~bin.mongorestore`
restores data to the :binary:`~bin.mongod` instance running on the
localhost interface on port ``27017``.
namespace pattern. The :option:`--nsInclude <mongorestore --nsInclude>`
statement specifies to only restore documents in the ``transactions``
database while :option:`--nsExclude <mongorestore --nsExclude>`
instructs :binary:`~bin.mongorestore` to exclude collections whose
names end with ``_dev``. :binary:`~bin.mongorestore` restores data to
the :binary:`~bin.mongod` instance running on the localhost interface
on port ``27017``.

.. code-block:: sh

Expand All @@ -256,15 +253,17 @@ Change Collections' Namespaces during Restore

.. versionadded:: 3.4

MongoDB 3.4 added the :option:`--nsFrom <mongorestore --nsFrom>` and :option:`--nsTo <mongorestore --nsTo>` options,
which enable you to change the namespace of a collection that you are
restoring. :option:`--nsFrom <mongorestore --nsFrom>` and :option:`--nsTo <mongorestore --nsTo>` support using
asterisks as wild cards *and* support using dollar signs to delimit
"wild card" variables to use in the replacement.
MongoDB 3.4 added the :option:`--nsFrom <mongorestore --nsFrom>` and
:option:`--nsTo <mongorestore --nsTo>` options, which enable you to
change the namespace of a collection that you are restoring.
:option:`--nsFrom <mongorestore --nsFrom>` and :option:`--nsTo
<mongorestore --nsTo>` support using asterisks as wild cards *and*
support using dollar signs to delimit "wild card" variables to use in
the replacement.

Consider a database ``data`` that you have exported to a ``dump/``
directory using :binary:`~bin.mongodump`. The ``data`` database contains the following
collections:
directory using :binary:`~bin.mongodump`. The ``data`` database
contains the following collections:

- ``sales_customer1``
- ``sales_customer2``
Expand All @@ -273,11 +272,13 @@ collections:
- ``users_customer2``
- ``users_customer3``

Using :option:`--nsFrom <mongorestore --nsFrom>` and :option:`--nsTo <mongorestore --nsTo>`, you can restore the data
into different namespaces. The following operation
Using :option:`--nsFrom <mongorestore --nsFrom>` and :option:`--nsTo
<mongorestore --nsTo>`, you can restore the data into different
namespaces. The following operation

- restores the ``sales_<customerName>`` collections in the ``data`` database to
``sales`` collections in the ``<customerName>`` database, and
- restores the ``sales_<customerName>`` collections in the ``data``
database to ``sales`` collections in the ``<customerName>`` database,
and

- restores the ``users_<customerName>`` collections to ``users``
collections in the ``<customerName>`` database.
Expand All @@ -289,12 +290,12 @@ into different namespaces. The following operation
Restore with Access Control
~~~~~~~~~~~~~~~~~~~~~~~~~~~

In the following example, :binary:`~bin.mongorestore` restores a database
dump located at ``/opt/backup/mongodump-2011-10-24``, to a database
running on port ``37017`` on the host
``mongodb1.example.net``. The :binary:`~bin.mongorestore` command authenticates to
the MongoDB instance using the username ``user`` and the
password ``pass``, as follows:
In the following example, :binary:`~bin.mongorestore` restores a
database dump located at ``/opt/backup/mongodump-2011-10-24``, to a
database running on port ``37017`` on the host
``mongodb1.example.net``. The :binary:`~bin.mongorestore` command
authenticates to the MongoDB instance using the username ``user`` and
the password ``pass``, as follows:

.. code-block:: sh

Expand Down Expand Up @@ -333,8 +334,8 @@ Restore a Database from Standard Input

.. versionadded:: 3.2

To restore from the standard input, run :binary:`~bin.mongorestore` with the
``archive`` option but *omit* the filename. For example:
To restore from the standard input, run :binary:`~bin.mongorestore`
with the ``archive`` option but *omit* the filename. For example:

.. code-block:: sh

Expand All @@ -350,9 +351,9 @@ Restore from Compressed Data
compressed files or data stream created by :binary:`~bin.mongodump`.

To restore from a dump directory that contains compressed files, run
:binary:`~bin.mongorestore` with the new ``--gzip`` option. For example, the
following operation restores the ``test`` database from the compressed
files located in the default ``dump`` directory:
:binary:`~bin.mongorestore` with the new ``--gzip`` option. For
example, the following operation restores the ``test`` database from
the compressed files located in the default ``dump`` directory:

.. code-block:: sh

Expand Down