Skip to content

Commit 9a99196

Browse files
mmaville-mdbmmaville-mdb
authored
MIG-3183-Conditional-Mapping-Rules (#147)
* MIG-3183-Conditional-Mapping-Rules * Refactoring pages, adding step outlines, created new step refs for rule filters and array conditions. * Fixing icon ref for ascending and descending button. * Cleaning up refs and step structure. * Adding entries to Mapping Rule Options page, additional clean-up. * Edits per internal feedback. Adding Mapping Rule Filters page, additional clean-up. * Edits per external feedback, fixing code example formatting. * Modying result codeblock formatting. * Fixing headings on Mapping Rule Filter page. * Formatting. * Formatting. * Formatting. * Updating Mapping Rule Filters example with table input and array output. Fixed filter example formatting. * Formatting example. * Reformatting step sections per feedback. Adding foreign key link advanced mapping option. Additional tweaks to steps. * Minor formatting fixes. * Formatting fixes. * Reformatting configure fields step. * Adding content back to configure field step. * Responding to feedback, single sourcing Configure fields step via include. * Removing column from example table and minor formatting fixes. * Adding a Merge fields into the parent step for embedded documents. * Removing old data from MongoDB output on mpaping rule filter example. * Tweaking example data for table. * Adding column widths. * Responding to internal feedback.
1 parent 3941069 commit 9a99196

9 files changed

+355
-36
lines changed

‎source/includes/fact-mapping-options-array-conditions.rst

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
Allows you to sort an embedded array and limit the amount of entries
2+
in that array. You can apply only a sort, only a limit, or both. When
3+
limiting to a single entry, there is the option to embed as a document
4+
instead of a single-element array by selecting the
5+
:guilabel:`Create an array of primitive values` :icon-fa5:`check-square`.
6+
7+
a. On the :guilabel:`Mappings` pane, click the :icon-fa5:`chevron-left`
8+
icon next to :guilabel:`Advanced settings`.
9+
#. Select the :guilabel:`Add array conditions` :icon-fa5:`check-square`
10+
icon.
11+
#. Enter a filter in the :guilabel:`Value expression` text box.
12+
#. In the :guilabel:`Sort by and order` heading, select the source
13+
field to sort on and toggle between :icon-lg:`SortAscending` for
14+
ascending and :icon-lg:`SortDescending` for descending order.
15+
#. Select a :guilabel:`Limit` option:
16+
17+
- :guilabel:`No limit`: No limit
18+
- :guilabel:`Limit number of rows`: Enter the maximum number of
19+
elements returned in the array. Default vaulue is ``10``.
20+
21+
.. note::
22+
23+
- Excluded fields cannot be sorted on. If a previously selected
24+
sorting field is excluded at a later point in time, the array
25+
condition is removed.
26+
27+
- If an array is modified during the CDC stage of a continuous
28+
job, sorting and limiting considers only the newly modified row
29+
and the pre-existing array elements, not the other values from
30+
the table which are not in the array.

‎source/includes/fact-mapping-options-configure-fields.rst

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
a. Define the options for the new mapping rule. When defining the
2+
mapping rule options, you can:
3+
4+
- Change the collection name.
5+
6+
To change the collection name, click the :guilabel:`Name` text
7+
box and enter the new name. Click outside of the text box to
8+
save the change. If the name is already in use, you receive a
9+
validation error.
10+
11+
- :ref:`Add Calculated Fields <rm-add-calculated-fields>` to
12+
create new fields based on relational source columns. See
13+
:ref:`Calculated Fields <rm-calculated-fields>` for more
14+
information.
15+
16+
- Change field names.
17+
18+
- Include or exclude a field from your sync job by clicking the
19+
:icon-fa5:`check-square` icon next to the field name.

‎source/includes/fact-mapping-options-foreign-key-link.rst

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
Allows you to select which foreign key to embed on when there is
2+
more than one foreign key between two tables. This option only
3+
appears for tables or collections with multiple foreign key links.
4+
5+
a. Select the name of the foreign key to embed on from the
6+
:guilabel:`Foreign key link` drop-down.

‎source/includes/fact-mapping-options-rule-filter.rst

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
Allows you to explicitly include rows where an expression returns
2+
``true``. This differs from :ref:`Table Filters <rm-table-filters>`,
3+
which filter based on an SQL query, and apply to all rows from a
4+
particular table. Mapping rule filters only apply to a specific
5+
mapping.
6+
7+
a. On the :guilabel:`Mappings` pane, click the :icon-fa5:`chevron-left` icon next to
8+
:guilabel:`Advanced settings`.
9+
#. Select the :guilabel:`Add mapping rule filter` :icon-fa5:`check-square` icon.
10+
#. Enter a valid JavaScript filter expression in the :guilabel:`Value expression` text box.

‎source/mapping-rules/mapping-rule-options/embedded-array.txt

Lines changed: 61 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -10,24 +10,74 @@ Embedded Array
1010
:depth: 1
1111
:class: singlecol
1212

13-
The :guilabel:`Embedded array` mapping rule is an advanced mapping rule option.
14-
It must be explicitly added to a mapping model.
13+
Use the :guilabel:`Embedded array` mapping rule to insert the
14+
values of the child table as array elements under documents in
15+
the collection mapped to the parent table. The
16+
:guilabel:`Embedded array` mapping rule is an advanced mapping
17+
rule option. It must be explicitly added to a mapping model.
1518

1619
About this Task
1720
---------------
1821

19-
The :guilabel:`Embedded array` mapping rule is available if the relational
20-
table it is mapping from is on the many side of the foreign key relationship.
22+
- The :guilabel:`Embedded array` mapping rule is available if the
23+
relational table it is mapping from is on the many side of the
24+
foreign key relationship.
2125

22-
The table containing the primary key must also be mapped to a
23-
collection in the MongoDB model.
26+
- The table containing the primary key must also be mapped to a
27+
collection in the MongoDB model.
2428

25-
Behavior
26-
--------
29+
Steps
30+
-----
2731

28-
Use the :guilabel:`Embedded array` mapping rule to insert the
29-
values of the child table as array elements under documents in
30-
the collection mapped to the parent table.
32+
.. procedure::
33+
:style: connected
34+
35+
.. step:: Select a table or collection
36+
37+
a. In the left :guilabel:`Schema model` pane, click on a
38+
collection under the :guilabel:`MongoDB` or
39+
:guilabel:`Relational` header.
40+
41+
This prompts the :guilabel:`Mappings` pane to open on the right of the screen.
42+
43+
.. step:: Create or edit a rule
44+
45+
a. From the :guilabel:`Mapping` screen, click :guilabel:`+ Add`
46+
to create a new mapping rule or click the
47+
:icon-fa5:`pencil-alt` icon to edit an existing rule.
48+
49+
b. Select :guilabel:`Embedded array` under :guilabel:`Migrate table as`.
50+
51+
.. step:: Configure the root path
52+
53+
a. Select the name of the :guilabel:`Source table` or
54+
:guilabel:`Parent collection` from the drop-down.
55+
56+
b. Configure the root path:
57+
58+
- :guilabel:`Prefix`: Specifies the rule hierarchy.
59+
60+
- :guilabel:`Field name`: Specifies the field containing the embedded array or document.
61+
62+
.. step:: (Optional) Configure multiple foreign keys
63+
64+
.. include:: /includes/fact-mapping-options-foreign-key-link.rst
65+
66+
.. step:: (Optional) Configure mapping rule filters
67+
68+
.. include:: /includes/fact-mapping-options-rule-filter.rst
69+
70+
.. step:: (Optional) Configure array conditions
71+
72+
.. include:: /includes/fact-mapping-options-array-conditions.rst
73+
74+
.. step:: Configure fields
75+
76+
.. include:: /includes/fact-mapping-options-configure-fields.rst
77+
78+
.. step:: Save and close
79+
80+
a. Click :guilabel:`Save And Close`.
3181

3282
Example
3383
-------

‎source/mapping-rules/mapping-rule-options/embedded-documents.txt

Lines changed: 68 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -10,33 +10,86 @@ Embedded Documents
1010
:depth: 1
1111
:class: singlecol
1212

13+
Use the :guilabel:`Embedded Documents` mapping rule to denormalize
14+
a foreign key relationship. With :guilabel:`Embedded Documents` you can
15+
nest child foreign key fields in a parent collection.
16+
17+
You have two main options when denormalizing your relational data
18+
with the :guilabel:`Embedded Documents` option:
19+
20+
- You can embed the child documents into the parent and designate a
21+
:guilabel:`Prefix` and :guilabel:`Field name`.
22+
23+
- You can merge into the parent, which represents the rows as
24+
fields at the parent level.
25+
1326
The :guilabel:`Embedded Documents` mapping rule is an advanced mapping
1427
rule option. It must be explicitly added to a mapping model.
1528

1629
About this Task
1730
---------------
1831

19-
The :guilabel:`Embedded Documents` mapping rule is available if the relational
20-
table it is mapping from is on the one side of the foreign key relationship.
32+
- The :guilabel:`Embedded Documents` mapping rule is available if the
33+
relational table it is mapping from is on the one side of the
34+
foreign key relationship.
2135

22-
The table containing the foreign key must also be mapped to a
23-
collection in the MongoDB model.
36+
- The table containing the foreign key must also be mapped to a
37+
collection in the MongoDB model.
2438

25-
Behavior
26-
--------
39+
Steps
40+
-----
2741

28-
Use the :guilabel:`Embedded Documents` mapping rule to denormalize
29-
a foreign key relationship. With :guilabel:`Embedded Documents` you can
30-
nest child foreign key fields in a parent collection.
42+
.. procedure::
43+
:style: connected
3144

32-
You have two main options when denormalizing your relational data
33-
with the :guilabel:`Embedded Documents` option:
45+
.. step:: Select a table or collection
3446

35-
- You can embed the child documents into the parent and designate a
36-
:guilabel:`Prefix` and :guilabel:`Field name` to embed fields as sub documents.
47+
a. In the left :guilabel:`Schema model` pane, click on a
48+
collection under the :guilabel:`MongoDB` or
49+
:guilabel:`Relational` header.
50+
51+
This prompts the :guilabel:`Mappings` pane to open on the right of the screen.
3752

38-
- You can merge into the parent, which represents the rows as
39-
fields at the parent level.
53+
.. step:: Create or edit a rule
54+
55+
a. From the :guilabel:`Mapping` screen, click :guilabel:`+ Add`
56+
to create a new mapping rule or click the
57+
:icon-fa5:`pencil-alt` icon to edit an existing rule.
58+
59+
b. Select :guilabel:`Embedded documents` under :guilabel:`Migrate table as`.
60+
61+
.. step:: Configure the root path
62+
63+
a. Select the name of the :guilabel:`Source table` or
64+
:guilabel:`Parent collection` from the drop-down.
65+
66+
b. Configure the root path:
67+
68+
- :guilabel:`Prefix`: Specifies the rule hierarchy.
69+
70+
- :guilabel:`Field name`: Specifies the field containing the embedded array or document.
71+
72+
.. step:: (Optional) Configure multiple foreign keys
73+
74+
.. include:: /includes/fact-mapping-options-foreign-key-link.rst
75+
76+
.. step:: (Optional) Merge fields into the parent
77+
78+
a. Merge table rows as fields at the parent collection
79+
level by clicking the :icon-fa5:`check-square` icon next to
80+
the field name.
81+
82+
.. step:: (Optional) Configure mapping rule filters
83+
84+
.. include:: /includes/fact-mapping-options-rule-filter.rst
85+
86+
.. step:: Configure fields
87+
88+
.. include:: /includes/fact-mapping-options-configure-fields.rst
89+
90+
.. step:: Save and close
91+
92+
a. Click :guilabel:`Save And Close`.
4093

4194
Examples
4295
--------

‎source/mapping-rules/mapping-rule-options/mapping-rule-filters.txt

Lines changed: 110 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,110 @@
1+
.. _rm-mapping-filters:
2+
3+
====================
4+
Mapping Rule Filters
5+
====================
6+
7+
.. contents:: On this page
8+
:local:
9+
:backlinks: none
10+
:depth: 1
11+
:class: singlecol
12+
13+
A mapping rule filter consists of a JavaScript
14+
expression which evaluates each row of input data. The
15+
column values from the row are available in the
16+
``columns["<column_name>"]`` object.
17+
18+
During a sync job, Relational Migrator evaluates the expression for
19+
each row:
20+
21+
- If the return value is ``true`` then the row is included in the
22+
migrated data.
23+
- If the return value is ``false`` then the row is excluded.
24+
- If the return value is not either, the row is excluded and an error
25+
is logged during migration.
26+
27+
Example
28+
-------
29+
30+
Below is an example of input documents and mapping rule JavaScript
31+
expressions that filter the data.
32+
33+
34+
Filter Based on Field Value
35+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
36+
37+
The following example filters a document based on a single field value.
38+
39+
Relational input:
40+
41+
.. list-table::
42+
:header-rows: 1
43+
:widths: 10 20 20 15 10 10 15
44+
45+
* - customer_id
46+
- company_name
47+
- address
48+
- city
49+
- postal_code
50+
- country
51+
- phone
52+
53+
54+
* - ALFKI
55+
- Alfreds Futterkiste
56+
- Obere Str. 57
57+
- Berlin
58+
- 12209
59+
- Germany
60+
- 030-0074321
61+
62+
* - ANATR
63+
- Ana Trujillo Emparedados y helados
64+
- Avda. de la Constitución 2222
65+
- México D.F.
66+
- 05021
67+
- Mexico
68+
- 5-555-4729
69+
70+
* - ANTON
71+
- Antonio Moreno Taquería
72+
- Mataderos 2312
73+
- México D.F.
74+
- 05023
75+
- Mexico
76+
- 5-555-3932
77+
78+
79+
Filter Expression:
80+
81+
.. code-block:: none
82+
83+
columns["country"] == "Mexico"
84+
85+
86+
MongoDB output:
87+
88+
.. code-block:: none
89+
:copyable: false
90+
91+
[
92+
{
93+
"customerId": "ANATR",
94+
"address": "Avda. de la Constitución 2222",
95+
"city": "México D.F.",
96+
"companyName": "Ana Trujillo Emparedados y helados",
97+
"country": "Mexico",
98+
"phone": "5-555-4729",
99+
"postalCode": "05021",
100+
},
101+
{
102+
"customerId": "ANTON",
103+
"address": "Mataderos 2312",
104+
"city": "México D.F.",
105+
"companyName": "Antonio Moreno Taquería",
106+
"country": "Mexico",
107+
"phone": "5-555-3932",
108+
"postalCode": "05023",
109+
}
110+
]

0 commit comments

Comments
 (0)