Skip to content

Commit 2749cfd

Browse files
jeff-allen-mongoDave
jeff-allen-mongo
and
Dave
committed
(DOCSP-19584): Tutorial for sharding a time series collection (#64)
* (DOCSP-19584): Tutorial for sharding a time series collection * word tweak * Add clarification * copy tweak * updates per copy review pt 1 * re-pushing * fix step file syntax * copy tweaks * more tweaks * remove some words * adjust release notes * add missing quotes around keys * tweak * Apply suggestions from code review Co-authored-by: Dave <[email protected]> * copy review part 2 * formatting tweak * reformatting * add limitations sub-heading * Updates per Arun's feedback * deleted wrong file * formatting * fix link formatting * Update 5.1 compat notes * fixing copy on compat notes page Co-authored-by: Dave <[email protected]>
1 parent bce0de9 commit 2749cfd

14 files changed

+316
-29
lines changed

‎snooty.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,7 @@ toc_landing_pages = [
7979
"/core/sharding-shard-key",
8080
"/core/storage-engines",
8181
"/core/timeseries-collections",
82+
"/core/timeseries-collections/timeseries-shard-collection",
8283
"/core/transactions",
8384
"/core/wiredtiger",
8485
"/core/zone-sharding",

‎source/core/timeseries-collections.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -378,3 +378,4 @@ Valid ``block_compressor`` options are:
378378
/core/timeseries/timeseries-secondary-index
379379
/core/timeseries/timeseries-migrate-data-into-timeseries-collection
380380
/core/timeseries/timeseries-build-materialized-views
381+
/core/timeseries/timeseries-shard-collection

‎source/core/timeseries/timeseries-limitations.txt

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -156,6 +156,16 @@ When using sharded time series collections, you cannot:
156156
- :dbcommand:`moveChunk`
157157
- :dbcommand:`splitChunk`
158158

159+
Shard Zones
160+
```````````
161+
162+
.. include:: /includes/time-series/fact-shard-zone-limitations.rst
163+
164+
Shard Keys
165+
``````````
166+
167+
.. include:: /includes/time-series/fact-shard-key-limitations.rst
168+
159169
Aggregation $out and $merge
160170
~~~~~~~~~~~~~~~~~~~~~~~~~~~
161171

‎source/core/timeseries/timeseries-shard-collection.txt

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
.. _manual-timeseries-shard-collection:
2+
3+
==============================
4+
Shard a Time Series Collection
5+
==============================
6+
7+
.. default-domain:: mongodb
8+
9+
.. contents:: On this page
10+
:local:
11+
:backlinks: none
12+
:depth: 1
13+
:class: singlecol
14+
15+
.. versionadded:: 5.1
16+
17+
Use this tutorial to shard a new or existing time series collection.
18+
19+
.. important::
20+
21+
Before completing this tutorial, review the :ref:`sharding
22+
limitations <time-series-limitations-sharding>` for time series
23+
collections.
24+
25+
Prerequisites
26+
-------------
27+
28+
To shard a time series collection, first:
29+
30+
1. :ref:`Deploy a sharded cluster <sharding-procedure-setup>` to host
31+
the database that contains your time series collection.
32+
33+
#. :ref:`Enable sharding for your database <sharding-setup-enable-sharding>`.
34+
35+
Procedure
36+
---------
37+
38+
Shard a New Time Series Collection
39+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
40+
41+
.. include:: /includes/steps/shard-new-tsc.rst
42+
43+
Shard an Existing Time Series Collection
44+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
45+
46+
.. include:: /includes/steps/shard-existing-tsc.rst
47+
48+
Additional Information
49+
----------------------
50+
51+
- :ref:`manual-timeseries-collection`
52+
53+
- :method:`sh.shardCollection()`
54+
55+
- :dbcommand:`shardCollection`

‎source/core/zone-sharding.txt

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -192,6 +192,11 @@ Shard Zone Boundaries
192192

193193
.. include:: /includes/fact-shard-ranges-inclusive-exclusive.rst
194194

195+
Time Series Collections
196+
~~~~~~~~~~~~~~~~~~~~~~~
197+
198+
.. include:: /includes/time-series/fact-shard-zone-limitations.rst
199+
195200
.. seealso::
196201

197202
:doc:`/tutorial/manage-shard-zone`

‎source/includes/5.1/5.1-release-notes-sharded-time-series.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3,5 +3,6 @@ MongoDB 5.1 provides support for sharded :ref:`time series collections
33

44
See:
55

6+
- :ref:`manual-timeseries-shard-collection`
67
- :dbcommand:`shardCollection`
78
- :ref:`Time Series Limitations <time-series-limitations-sharding>`

‎source/includes/steps-shard-existing-tsc.yaml

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
title: Connect to your sharded cluster.
2+
ref: new-sharded-tsc-connect
3+
stepnum: 1
4+
level: 4
5+
content: |
6+
Connect :binary:`~bin.mongosh` to the :binary:`~bin.mongos` for your
7+
sharded cluster. Specify the ``host`` and ``port`` on which the
8+
``mongos`` is running:
9+
10+
.. code-block:: javascript
11+
12+
mongosh --host <hostname> --port <port>
13+
---
14+
title: Shard the collection.
15+
ref: new-sharded-tsc-create
16+
stepnum: 2
17+
level: 4
18+
content: |
19+
Use the :method:`~sh.shardCollection()` method to shard the
20+
collection.
21+
22+
Consider a time series collection with the following properties:
23+
24+
.. code-block:: javascript
25+
26+
db.createCollection(
27+
"deliverySensor",
28+
{
29+
timeseries: {
30+
timeField: "timestamp",
31+
metaField: "metadata",
32+
granularity: "minutes"
33+
}
34+
}
35+
)
36+
37+
A sample document from the collection resembles:
38+
39+
.. code-block:: javascript
40+
41+
db.deliverySensor.insertOne( {
42+
"metadata": { "location": "USA", "vehicle": "truck" },
43+
"timestamp": ISODate("2021-08-21T00:00:10.000Z"),
44+
"speed": 50
45+
} )
46+
47+
To shard the collection, run the following command:
48+
49+
.. code-block:: javascript
50+
51+
sh.shardCollection( "test.deliverySensor", { "metadata.location": 1 } )
52+
53+
In this example, :method:`sh.shardCollection()`:
54+
55+
- Shards an existing time series collection named ``deliverySensor`` on
56+
the ``test`` database.
57+
58+
- Specifies the ``metadata.location`` field as the :ref:`shard key
59+
<shard-key>`. ``location`` is a sub-field of the collection's
60+
``metaField``.
61+
62+
When the collection you specify to :method:`sh.shardCollection()` is
63+
a time series collection, you do not need to specify the
64+
:ref:`timeseries <method-sharded-time-series-collection-options>`
65+
option.
66+
67+
...

‎source/includes/steps-shard-new-tsc.yaml

Lines changed: 59 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,59 @@
1+
title: Connect to your sharded cluster.
2+
ref: new-sharded-tsc-connect
3+
stepnum: 1
4+
level: 4
5+
content: |
6+
Connect :binary:`~bin.mongosh` to the :binary:`~bin.mongos` for your
7+
sharded cluster. Specify the ``host`` and ``port`` on which the
8+
``mongos`` is running:
9+
10+
.. code-block:: javascript
11+
12+
mongosh --host <hostname> --port <port>
13+
---
14+
title: Create the collection.
15+
ref: new-sharded-tsc-create
16+
stepnum: 2
17+
level: 4
18+
content: |
19+
Use the :method:`~sh.shardCollection()` method with the :ref:`timeseries
20+
<method-sharded-time-series-collection-options>` option.
21+
22+
For example:
23+
24+
.. code-block:: javascript
25+
26+
sh.shardCollection(
27+
"test.weather",
28+
{ "metadata.sensorId": 1 },
29+
{
30+
timeseries: {
31+
timeField: "timestamp",
32+
metaField: "metadata",
33+
granularity: "hours"
34+
}
35+
}
36+
)
37+
38+
In this example, :method:`sh.shardCollection()`:
39+
40+
- Shards a new time series collection named ``weather`` on the
41+
``test`` database.
42+
43+
- Specifies the ``metadata.sensorId`` field as the :ref:`shard key
44+
<shard-key>`.
45+
46+
- Specifies a ``granularity`` of hours.
47+
48+
The following document contains the appropriate metadata for the
49+
collection:
50+
51+
.. code-block:: javascript
52+
53+
db.weather.insertOne( {
54+
"metadata": { "sensorId": 5578, "type": "temperature" },
55+
"timestamp": ISODate("2021-05-18T00:00:00.000Z"),
56+
"temp": 12
57+
} )
58+
59+
...

‎source/includes/time-series/fact-shard-key-limitations.rst

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
When sharding time series collections, you can only specify
2+
the following fields in the shard key:
3+
4+
- The ``metaField``
5+
- Sub-fields of ``metaField``
6+
- The ``timeField``
7+
8+
You may specify combinations of these fields in the shard key. No other
9+
fields, including ``_id``, are allowed in the shard key pattern.
10+
11+
When you specify the shard key:
12+
13+
- ``metaField`` can be either a:
14+
15+
- :ref:`Hashed shard key <sharding-hashed-sharding>`
16+
- :ref:`Ranged shard key <sharding-ranged>`
17+
18+
- ``timeField`` must be:
19+
20+
- A :ref:`ranged shard key <sharding-ranged>`
21+
- At the end of the shard key pattern
22+
23+
.. tip::
24+
25+
Avoid specifying **only** the ``timeField`` as the shard key. Since
26+
the ``timeField`` :ref:`increases monotonically
27+
<shard-key-monotonic>`, it may result in all writes appearing on a
28+
single chunk within the cluster. Ideally, data is evenly distributed
29+
across chunks.
30+
31+
To learn how to best choose a shard key, see:
32+
33+
- :ref:`sharding-shard-key-requirements`
34+
- `MongoDB Blog: On Selecting a Shard Key for MongoDB
35+
<https://www.mongodb.com/blog/post/on-selecting-a-shard-key-for-mongodb?tck=docs_server>`__.

‎source/includes/time-series/fact-shard-zone-limitations.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
MongoDB does not support creating zones for sharded
2+
:ref:`time series collections <manual-timeseries-collection>`.

‎source/includes/time-series/fact-timeseries-param-desc.rst

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
Optional. Specify this option to create a new sharded :ref:`time series
2+
collection <manual-timeseries-collection>`.
3+
4+
To shard an existing time series collection, omit this parameter.
5+
6+
When the collection specified to ``shardCollection`` is a time series
7+
collection and the ``timeseries`` option is not specified, MongoDB uses
8+
the values that define the existing time series collection to populate
9+
the ``timeseries`` field.

‎source/reference/command/shardCollection.txt

Lines changed: 8 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -177,30 +177,24 @@ Definition
177177

178178
- .. _cmd-shard-collection-timeseries:
179179

180-
Optional. Specify this option to create a new sharded
181-
:ref:`time series collection <manual-timeseries-collection>`.
182-
183-
To shard an existing time series collection, omit this
184-
parameter. When you omit this parameter and specify a time
185-
series collection in the ``shardCollection`` parameter,
186-
MongoDB automatically uses the values from the time series
187-
collection as the values for the ``timeseries`` field.
180+
.. include:: /includes/time-series/fact-timeseries-param-desc.rst
188181

189182
For detailed syntax, see
190-
:ref:`sharded-time-series-collection-options`.
183+
:ref:`cmd-sharded-time-series-collection-options`.
191184

192185
.. versionadded:: 5.1
193186

194-
.. _sharded-time-series-collection-options:
187+
.. _cmd-sharded-time-series-collection-options:
195188

196189
Time Series Options
197190
~~~~~~~~~~~~~~~~~~~
198191

199192
.. versionadded:: 5.1
200193

201-
Specify the :ref:`timeseries <cmd-shard-collection-timeseries>` option
202-
to :dbcommand:`shardCollection` to create a new sharded
203-
:ref:`time series collection <manual-timeseries-collection>`.
194+
To create a new :ref:`time series collection
195+
<manual-timeseries-collection>` that is sharded, specify the
196+
:ref:`timeseries <cmd-shard-collection-timeseries>` option to
197+
:dbcommand:`shardCollection`.
204198

205199
The :ref:`timeseries <cmd-shard-collection-timeseries>` option takes
206200
the following fields:
@@ -225,11 +219,6 @@ the following fields:
225219
- string
226220
- .. include:: /includes/time-series/fact-granularity-description.rst
227221

228-
* - ``bucketMaxSpanSeconds``
229-
- integer
230-
- Optional. The maximum range of time values for a bucket,
231-
in seconds.
232-
233222
Considerations
234223
--------------
235224

@@ -256,17 +245,7 @@ avoid scalability and perfomance issues.
256245
Shard Keys on Time Series Collections
257246
`````````````````````````````````````
258247

259-
When sharding time series collections, you can only specify the
260-
``metaField`` (or sub-fields of ``metaField``), ``timeField``, or both
261-
in the shard key. No other fields, including ``_id``, are allowed in the
262-
shard key pattern.
263-
264-
- ``metaField`` can be either a :ref:`hashed shard key
265-
<sharding-hashed-sharding>` or a :ref:`ranged shard key
266-
<sharding-ranged>`.
267-
268-
- ``timeField`` can only be a :ref:`ranged shard key
269-
<sharding-ranged>` and must be at the end of the shard key pattern.
248+
.. include:: /includes/time-series/fact-shard-key-limitations.rst
270249

271250
.. _hashed-shard-keys:
272251

0 commit comments

Comments
 (0)