Skip to content

Commit 6e7769c

Browse files
jmd-mongojmd-mongo
authored
Merge pull request #99 from ianf-mongodb/DOCSP-37407
DOCSP-37407 Merge Prerequisite Content to Master
2 parents 3d4fb2a + aaecff4 commit 6e7769c

12 files changed

+790
-2
lines changed

‎snooty.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -33,11 +33,11 @@ toc_landing_pages = [
3333
"code-generation",
3434
"code-generation/generate-app-code",
3535
"code-generation/query-converter",
36-
"diagrams/manage-diagrams"
36+
"diagrams/manage-diagrams",
37+
"prerequisites"
3738
]
3839

3940
[constants]
4041
ddl = ":abbr:`DDL (Data Definition Language)`"
4142
job = "sync job" #this will be updated to migration job in the near future.
4243
job_plural = "sync jobs" #this will be updated to migration jobs in the near future.
43-

‎source/includes/fact-data-prep-oracle-step1.rst

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
The following code creates a new Oracle service account
2+
for Relational Migrator to connect to the Oracle
3+
instance. Alternatively, you can use an existing Oracle
4+
service account to connect to Relational Migrator with
5+
the appropriate permissions.
6+
7+
a. Create a service account:
8+
9+
.. code-block:: sql
10+
:copyable: true
11+
12+
CREATE USER '<user>'@'localhost' IDENTIFIED BY '<password>';
13+
14+
#. Grant select permissions to the service account:
15+
16+
The required permission for the service account depend on whether
17+
the tables are owned by the service account used to run the sync job.
18+
To check table ownership run the following query:
19+
20+
.. code-block:: sql
21+
:copyable: true
22+
23+
SELECT TABLE_NAME, OWNER
24+
FROM ALL_TABLES
25+
WHERE TABLE_NAME ='<table_name>'
26+
ORDER BY OWNER, TABLE_NAME;
27+
28+
If the service account *is* the table owner:
29+
30+
.. code-block:: sql
31+
:copyable: true
32+
33+
GRANT CREATE SESSION TO <user>;
34+
GRANT SELECT ON V_$DATABASE TO <user>;
35+
36+
If the service account *is not* the table owner:
37+
38+
.. code-block:: sql
39+
:copyable: true
40+
41+
GRANT CREATE SESSION TO <user>;
42+
GRANT SELECT_CATALOG_ROLE TO <user>;
43+
GRANT SELECT <table> TO <user>;
44+
GRANT SELECT ON V_$DATABASE TO <user>;
45+
GRANT FLASHBACK ON <table> TO user;

‎source/includes/fact-enable-cdc-database.rst

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,25 @@
1+
To enable CDC at the database level
2+
use the ``sys.sp_cdc_enable_db`` stored procedure.
3+
4+
The code blocks below are a sample of the code
5+
automatically-generated by Relational Migrator.
6+
You can run the code manually by replacing the
7+
database name for ``MyDB``:
8+
9+
.. code-block:: sql
10+
:copyable: true
11+
12+
USE MyDB
13+
GO
14+
EXEC sys.sp_cdc_enable_db
15+
GO
16+
17+
For SQL Server instances hosted on AWS RDS:
18+
19+
.. code-block:: sql
20+
:copyable: true
21+
22+
USE MyDB
23+
GO
24+
EXEC msdb.dbo.rds_cdc_enable_db 'MyDB';
25+
GO

‎source/includes/fact-my-sql-setup-user-permission-step.rst

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
.. step:: (Optional) Set up user permissions
2+
3+
The following code creates a new MySQL service account
4+
for Relational Migrator to connect to the MySQL
5+
instance. Alternatively, you can use an existing MySQL service
6+
account to connect to Relational Migrator with the appropriate
7+
permissions.
8+
9+
a. Create a service account:
10+
11+
.. code-block:: sql
12+
:copyable: true
13+
14+
CREATE USER 'user'@'localhost' IDENTIFIED BY 'password';
15+
16+
#. Grant the required permissions to the service account:
17+
18+
.. code-block:: sql
19+
:copyable: true
20+
21+
GRANT SELECT, RELOAD, SHOW DATABASES, REPLICATION SLAVE, REPLICATION CLIENT
22+
ON *.*
23+
TO 'user'@'%';
24+
25+
#. Apply the user privilege changes:
26+
27+
.. code-block:: sql
28+
:copyable: true
29+
30+
FLUSH PRIVILEGES;

‎source/includes/fact-short-sync-job-desc.rst

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
- Snapshot sync jobs migrate all the data and then stops.
2+
- Continuous sync job run a snapshot and then enter a CDC stage to
3+
continuously replicate data changes.

‎source/index.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -153,6 +153,7 @@ Explore libraries and tools for MongoDB.
153153
/supported-databases
154154
/migration-scenarios
155155
/installation
156+
/prerequisites
156157
/projects/projects
157158
/mapping-rules/mapping-rules
158159
/table-filters/table-filters

‎source/prerequisites.txt

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
.. _rm-prerequisites:
2+
3+
=============
4+
Prerequisites
5+
=============
6+
7+
.. contents:: On this page
8+
:local:
9+
:backlinks: none
10+
:depth: 1
11+
:class: singlecol
12+
13+
The following pages outline the details and specifics of prepping your
14+
database for use with Relational Migrator. The instructions for these
15+
pages reiterate the steps and procedures of the SQL scripts that are
16+
automatically generated by Relational Migrator when you create your
17+
first sync job:
18+
19+
.. list-table::
20+
:header-rows: 1
21+
:stub-columns: 1
22+
:widths: 30 40 40
23+
24+
* - Database
25+
- Snapshot Sync Job Support
26+
- Continuous Sync Job Support
27+
28+
* - :ref:`MySQL <rm-prereq-mysql>`
29+
- :icon-lg:`Checkmark`
30+
- :icon-lg:`Checkmark`
31+
32+
* - :ref:`Oracle <rm-prereq-oracle>`
33+
- :icon-lg:`Checkmark`
34+
- :icon-lg:`Checkmark`
35+
36+
* - :ref:`PostGreSQL <rm-prereq-postgres>`
37+
- :icon-lg:`Checkmark`
38+
- :icon-lg:`Checkmark`
39+
40+
* - :ref:`SQL Server <rm-prereq-sqlserver>`
41+
- :icon-lg:`Checkmark`
42+
- :icon-lg:`Checkmark`
43+
44+
* - :ref:`Sybase <rm-prereq-sybase>`
45+
- :icon-lg:`Checkmark`
46+
-
47+
48+
.. toctree::
49+
:titlesonly:
50+
:hidden:
51+
52+
/prerequisites/my-sql
53+
/prerequisites/oracle
54+
/prerequisites/postgres
55+
/prerequisites/sql-server
56+
/prerequisites/sybase

‎source/prerequisites/my-sql.txt

Lines changed: 180 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,180 @@
1+
.. _rm-prereq-mysql:
2+
3+
===========================================
4+
Configure Migration Prerequisites for MySQL
5+
===========================================
6+
7+
.. contents:: On this page
8+
:local:
9+
:backlinks: none
10+
:depth: 1
11+
:class: singlecol
12+
13+
To run sync jobs from a MySQL source database, the database may require
14+
some configuration changes. If Relational Migrator determines the
15+
database needs configuration changes, it automatically generates a
16+
SQL script with the required changes. It is recommended to have a
17+
Database Administrator (DBA) review the commands in this script and
18+
perform their execution on the database server. The
19+
MySQL Server configurations depend on the type of sync job:
20+
21+
.. include:: /includes/fact-short-sync-job-desc.rst
22+
23+
For details on supported versions of MySQL, see
24+
:ref:`supported-databases`.
25+
26+
Steps
27+
-----
28+
29+
30+
.. tabs::
31+
32+
.. tab:: Snapshot Jobs
33+
:tabid: enable-snapshot-jobs
34+
35+
.. procedure::
36+
:style: normal
37+
38+
.. include:: /includes/fact-my-sql-setup-user-permission-step.rst
39+
40+
.. tab:: Continuous Jobs
41+
:tabid: enable-continuous-jobs
42+
43+
Running continuous jobs on Relational Migrator
44+
requires the `binary log <https://dev.mysql.com/doc/refman/8.0/en/binary-log.html>`__
45+
to be enabled on your MySQL instance. The binary log (Binlog)
46+
records all operations in the order they are committed to the
47+
database.
48+
49+
.. procedure::
50+
:style: normal
51+
52+
.. include:: /includes/fact-my-sql-setup-user-permission-step.rst
53+
54+
.. step:: (Optional) Manually verify Binlog is enabled
55+
56+
Relational Migrator automatically checks this setting for
57+
you. To manually check the if the ``Binlog`` option is
58+
enabled, use the queries below for your version of MySQL:
59+
60+
.. note::
61+
62+
``Binlog`` is automatically enabled by default
63+
on MySQL ``8.x`` versions.
64+
65+
.. tabs::
66+
67+
.. tab:: MySql 8.x
68+
:tabid: mysql-8x
69+
70+
.. code-block:: sql
71+
:copyable: true
72+
73+
SELECT variable_value as "BINARY LOGGING STATUS (log-bin) ::"
74+
75+
FROM performance_schema.global_variables WHERE variable_name='log_bin';
76+
77+
.. tab:: MySql 5.x
78+
:tabid: mysql-5x
79+
80+
.. code-block:: sql
81+
:copyable: true
82+
83+
SELECT variable_value as "BINARY LOGGING STATUS (log-bin) ::"
84+
85+
FROM information_schema.global_variables WHERE variable_name='log_bin';
86+
87+
.. step:: Locate and update the MySQL configuration file
88+
89+
a. Run the following SQL query to get the ``server_id``
90+
value for your MySQL instance:
91+
92+
.. tabs::
93+
94+
.. tab:: MySql 8.x
95+
:tabid: mysql-8x
96+
97+
.. code-block:: sql
98+
:copyable: true
99+
100+
SELECT variable_value
101+
FROM
102+
performance_schema.global_variables
103+
WHERE variable_name='server_id';
104+
105+
.. tab:: MySql 5.x
106+
:tabid: mysql-5x
107+
108+
.. code-block:: sql
109+
:copyable: true
110+
111+
SELECT variable_value
112+
FROM
113+
information_schema.global_variables
114+
WHERE variable_name='server_id';
115+
116+
#. Locate the config file for your MySQL instance by running
117+
the following ``mysqld`` command in your terminal:
118+
119+
.. tabs::
120+
121+
.. tab:: Windows
122+
:tabid: windows
123+
124+
.. code-block::
125+
:copyable: true
126+
127+
mysql --help | findstr cnf
128+
129+
.. tab:: MacOS / Linux
130+
:tabid: macos-linux
131+
132+
.. code-block::
133+
:copyable: true
134+
135+
mysql --help | grep cnf
136+
137+
#. Under the ``[mysqld]`` section of your MySQL
138+
configuration file add the following lines. Replace
139+
the ``XXXXX`` value with the ``server_id`` from the
140+
previous query:
141+
142+
.. tabs::
143+
144+
.. tab:: MySql 8.x
145+
:tabid: mysql-8x
146+
147+
.. code-block::
148+
:copyable: true
149+
150+
server-id = XXXXX
151+
log_bin = mysql-bin
152+
binlog_format = ROW
153+
binlog_row_image = FULL
154+
binlog_expire_logs_seconds = 864000
155+
156+
.. tab:: MySql 5.x
157+
:tabid: mysql-5x
158+
159+
.. code-block::
160+
:copyable: true
161+
162+
server-id = XXXXX
163+
log_bin = mysql-bin
164+
binlog_format = ROW
165+
binlog_row_image = FULL
166+
expire_log_days = 10
167+
168+
.. note::
169+
170+
If you're running MySQL on AWS RDS and `automated backups
171+
<https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_WorkingWithAutomatedBackups.html>`__
172+
are not enabled, ``Binlog`` will be disabled, even if
173+
the values are set in the configuration file.
174+
175+
Learn More
176+
----------
177+
178+
Relational Migrator relies on the open-source Debezium connector to
179+
capture row-level changes. For more details, see
180+
`Debezium MySQL <https://debezium.io/documentation/reference/stable/connectors/mysql.html>`__.

0 commit comments

Comments
 (0)