Skip to content

Commit f3a2b7d

Browse files
kennethdyertfogomdb-ashley
authored
DOCSP-31391 1.5 release notes (#141)
* DOCSP-31391 Release Notes for 1.5.0 * DOCSP-29500 Clarify oplog Sizing (#131) * DOCSP-29500 clarify oplog sizing * fixes build issue * Refactors text * Refactors text * Refactors text * Fixes per Dave * Fixes per Evgeni * fixes per Alex * fixes per Alex --------- Co-authored-by: Tim Fogarty <[email protected]> * Adds to release note * Adds bugfixes * Spellcheck * Adds minimum supported version * Fixes per Sarah * Fixes per Sarah * Fixes per Sarah * Fixes build issue * Fixes per Michael * Fixes per Evgeni * Fixes per Evgeni * Fixes per Michael * Fixes per Evgeni * Fixes per Sarah * Fixes per Ashley Co-authored-by: Ashley Brown <[email protected]> --------- Co-authored-by: Tim Fogarty <[email protected]> Co-authored-by: Ashley Brown <[email protected]>
1 parent 183db4d commit f3a2b7d

File tree

6 files changed

+101
-65
lines changed

6 files changed

+101
-65
lines changed

‎source/faq.txt

Lines changed: 3 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -31,19 +31,10 @@ clusters.
3131
Should I increase the size of the ``oplog`` in the source cluster?
3232
------------------------------------------------------------------
3333

34-
.. include:: /includes/fact-oplog-background
35-
36-
The proper ``oplog`` size depends on system hardware, network speed,
37-
and other factors including system workload. However, assuming network
38-
transfer speeds of 30-50GB per hour, a rough formula to estimate the
39-
required ``oplog`` size is:
40-
41-
.. code-block:: shell
42-
43-
minimumRetentionHours = dataSizeInGB / 30
34+
The :term:`oplog` is a capped collection that keeps a rolling
35+
record of all operations that modify the data stored in your databases.
4436

45-
To estimate the size of ``oplog`` needed for initial synchronization,
46-
see: :ref:`c2c-oplog-sizing`.
37+
.. include:: /includes/fact-oplog-background
4738

4839
To learn more about how to increase the size of the ``oplog``, see:
4940
:ref:`tutorial-change-oplog-size`.

‎source/includes/fact-oplog-background.rst

Lines changed: 15 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,17 @@
1-
The :term:`oplog` in the source cluster must be large enough to track
2-
events that happen during the time it takes to complete the initial
3-
sync to the destination cluster.
1+
2+
``mongosync`` applies operations in the ``oplog`` on the source cluster
3+
to the data on the destination cluster. When operations
4+
that ``mongosync`` has not applied roll off the ``oplog``
5+
on the source cluster, the sync fails and ``mongosync`` exits.
6+
7+
During the initial sync, ``mongosync`` may apply operations at a slower
8+
rate due to the load imposed by copying documents concurrently.
9+
After ``mongosync`` completes the initial sync, it applies changes
10+
faster and is more likely to maintain a position in the ``oplog``
11+
that is close tothe real-time writes occuring on the source closter.
412

513
If you anticipate syncing a large data set, or if you plan to pause
6-
synchronization for an extended period of time, increase the size of the
7-
replica set ``oplog`` in the source cluster.
14+
synchronization for an extended period of time, you might exceed the
15+
:term:`oplog window`. Use the :setting:`~replication.oplogSizeMB` setting
16+
to increase the size of the ``oplog`` on the source cluster.
17+

‎source/includes/opts/verbosity.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
.. reference/configuration.txt
22
.. reference/mongosync.txt
33
4-
*Default*: ``INFO``
4+
*Default*: ``DEBUG``
55

66
Sets the verbosity level to use in log messages.
77
{+c2c-product-name+} logs all messages at the specified level and

‎source/reference/oplog-sizing.txt

Lines changed: 23 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
.. _c2c-oplog-sizing:
22

33
================
4-
``oplog`` Sizing
4+
oplog Sizing
55
================
66

77
.. default-domain:: mongodb
@@ -16,17 +16,18 @@ The :ref:`mongosync <c2c-mongosync>` program uses :ref:`change streams
1616
<changeStreams>` to synchronize data between source and destination
1717
clusters. ``mongosync`` does not access the :term:`oplog` directly,
1818
but when a change stream returns events from the past, the events must
19-
be within the :term:`oplog` time range.
19+
be within the ``oplog`` time range.
2020

21-
.. include:: /includes/fact-oplog-background.txt
2221

23-
Estimate ``oplog`` Size Needed for Initial Sync
22+
.. include:: /includes/fact-oplog-background
23+
24+
Monitor oplog Size Needed for Initial Sync
2425
-----------------------------------------------
2526

2627
.. procedure::
2728
:style: normal
2829

29-
.. step:: Determine the ``oplog`` Time Span
30+
.. step:: Determine oplog Window
3031

3132
To get the difference in seconds between the first and last entry
3233
in the ``oplog`` run :method:`db.getReplicationInfo()`. If you
@@ -40,55 +41,30 @@ Estimate ``oplog`` Size Needed for Initial Sync
4041
cluster. If there are multiple shards, the smallest number is the
4142
minimum ``oplog`` window.
4243

43-
.. step:: Estimate Copy Rate During Synching
44-
45-
.. _c2c-step-est-size:
46-
47-
To gather performance data while synching, start the sync process
48-
and monitor how fast data is transferred between clusters.
49-
50-
To start syncing, run the :ref:`/start <c2c-api-start>`
51-
command.
44+
.. step:: Determine mongosync Replication Lag
5245

53-
To get the ``copy_rate``:
54-
55-
- run the :ref:`/progress <c2c-api-progress>` command to get
56-
``estimatedCopiedBytes_time01``
57-
- wait a second or two
58-
- run the :ref:`/progress <c2c-api-progress>` command to get
59-
``estimatedCopiedBytes_time02``
60-
61-
The ``copy_rate`` is:
62-
63-
.. code-block:: shell
64-
65-
copy_rate = ( estimatedCopiedBytes_time02 - estimatedCopiedBytes_01) / time_between_requests
66-
67-
.. step:: Estimate Copy Time
68-
69-
Estimate the time needed to copy the entire collection. The
70-
estimated copy time is:
71-
72-
.. code-block:: shell
46+
To get the ``lagTimeSeconds`` value, run the
47+
:ref:`/progress <c2c-api-progress>` command.
48+
The lag time is the time in seconds between the
49+
last event applied by ``mongosync`` and time of the current
50+
latest event on the source cluster.
7351

74-
estimatedCopyTime = estimatedTotalBytes / copy_rate
52+
It is a measure of how far behind the source cluster ``mongosync`` is.
7553

76-
.. step:: Validate ``oplog`` Size
54+
.. step:: Validate oplog Size
7755

78-
If the estimated time is larger than the minimum oplog window you
79-
must cancel synchronization. Before restarting, make one of the
80-
following changes:
56+
If the lag time approaches the minimum ``oplog`` window, make
57+
one of the following changes:
8158

82-
- Increase the oplog window. Use :dbcommand:`replSetResizeOplog`
83-
to set ``minRetentionHours`` greater than the estimated copy
84-
time.
85-
- Scale up the ``mongosync`` instance. Add cpu or memory to scale
59+
- Increase the ``oplog`` window. Use :dbcommand:`replSetResizeOplog`
60+
to set ``minRetentionHours`` greater than the current ``oplog``
61+
window.
62+
- Scale up the ``mongosync`` instance. Add CPU or memory to scale
8663
up the ``mongosync`` node so that it has a higher copy rate.
8764

8865
.. note::
8966

90-
The copy rate may vary during synchronization. To monitor progress,
91-
repeat the :ref:`steps to estimate the copy rate
92-
<c2c-step-est-size>` and verify that the copy rate stays about the
93-
same.
67+
The :term:`oplog` window and rate of change for replication lag may vary
68+
during synchronization. Repeat these steps during a migration to monitor the
69+
progress.
9470

‎source/release-notes.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@ Release Notes
88
.. toctree::
99
:titlesonly:
1010

11+
/release-notes/1.5
1112
/release-notes/1.4
1213
/release-notes/1.3
1314
/release-notes/1.2

‎source/release-notes/1.5.txt

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
.. _c2c-release-notes-1.5:
2+
3+
===================================
4+
Release Notes for mongosync 1.5
5+
===================================
6+
7+
8+
.. default-domain:: mongodb
9+
10+
.. contents:: On this page
11+
:local:
12+
:backlinks: none
13+
:depth: 2
14+
:class: singlecol
15+
16+
.. _1.5.0-c2c-release-notes:
17+
18+
1.5.0 Release
19+
-------------
20+
21+
**July 25, 2023**
22+
23+
Oplog Rollover Resiliency
24+
~~~~~~~~~~~~~~~~~~~~~~~~~
25+
26+
Starting in 1.5.0, ``mongosync`` begins to apply changes while the
27+
initial sync is still in progress. By starting to apply changes earlier,
28+
``mongosync`` maintains a more recent position in the :term:`oplog`.
29+
This adds resiliency to long-running operations, mitigates the risk
30+
of ``oplog`` rollover, and significantly lowers the risk of restarting
31+
the sync.
32+
33+
Logging Level
34+
~~~~~~~~~~~~~
35+
36+
Starting in 1.5.0, the default logging level is ``DEBUG``. To change
37+
the logging level, see the :setting:`verbosity` setting.
38+
39+
40+
Bug Fixes
41+
---------
42+
43+
- Fixes an issue where very large collections could timeout during
44+
``mongosync`` initialization.
45+
46+
- Fixes an issue where ``mongosync`` could incorrectly report indexes
47+
as mismatched.
48+
49+
- Fixes an issue where the :ref:`/start <c2c-api-start>` endpoint would not return
50+
an error when passed a ``sharding`` object without the ``shardingEntries``
51+
key.
52+
53+
- Changes telemetry to send less metadata to Segment.
54+
55+
Minimum Supported Version
56+
-------------------------
57+
58+
In 1.5, the minimum supported version of MongoDB is 6.0.8.

0 commit comments

Comments
 (0)