Skip to content

Update Convert Replica Set to Replicated Shard Cluster Tutorial #1894

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Update Convert Replica Set to Replicated Shard Cluster Tutorial #1894

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
51 changes: 51 additions & 0 deletions source/includes/steps-convert-replica-set-add-shards.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
title:
text: Connect to the ``mongos``.
character: "`"
stepnum: 1
ref: connect-mongos
action:
language: javascript
code: |
mongo mongodb6.example.net:27017/admin
---
title:
text: Add a shard with ``mongos``.
character: "`"
stepnum: 2
ref: add-mongos-shard
action:
- pre: "Add a shard to the cluster with the :dbcommand:`addShard` command:"
language: javascript
code: |
sh.addShard( "rs0/mongodb0.example.net:27017,mongodb1.example.net:27017,mongodb2.example.net:27017" )
- pre: "A confirmation message like this will appear:"
language: javascript
code: |
{ "shardAdded" : "rs0", "ok" : 1 }
---
title:
text: Verify the shard is properly configured.
character: "`"
stepnum: 3
ref: verify-shard-configuration
action:
- pre: "Use this command:"
language: javascript
code: |
db.runCommand( { listShards : 1 } )
- pre: "A confirmation message like this will appear to list one or more shards:"
language: javascript
code: |
{
"shards" : [
{
"_id" : "rs0",
"host" : "rs0/mongodb0.example.net:27017,mongodb1.example.net:27017,mongodb2.example.net:27017"
},
],
"ok" : 1
}
post: |
Repeat these steps for each shard added to the cluster.
...
149 changes: 149 additions & 0 deletions source/includes/steps-convert-replica-set-enable-sharding.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
title:
text: Enabling sharding on the database level.
character: "`"
stepnum: 1
ref: enable-sharding-db-level
action:
- pre: |
Type the :dbcommand:`enableSharding` command to enable sharding on
the ``test`` database:
language: sh
code: |
db.runCommand( { enableSharding : "test" } )
- pre: |
This command returns the following output:
language: sh
code: |
{ "ok" : 1 }
---
title:
text: Create an index on the shard key.
character: "`"
stepnum: 2
ref: enable-sharding-create-index
pre: |
MongoDB uses the shard key to distribute documents between shards. Once
selected, you cannot change the shard key. Good shard keys:

- have values that are evenly distributed among all documents,

- group documents that are often accessed at the same time into contiguous chunks, and

- allow for effective distribution of activity among shards.

To learn more, see the :ref:`Shard Key Overview <sharding-shard-key>` and
:ref:`Shard Key <sharding-internals-shard-keys>` sections.

Create the index with the following command:
action:
language: sh
code: |
use test
db.test_collection.ensureIndex( { number : 1 } )
---
title:
text: Shard the collection.
character: "`"
stepnum: 3
ref: enable-sharding-collection
action:
- pre: |
Type these commands:
language: sh
code: |
use admin
db.runCommand( { shardCollection : "test.test_collection", key : { "number" : 1 } } )
- pre: |
These commands return this output:
language: sh
code: |
{ "collectionsharded" : "test.test_collection", "ok" : 1 }
post: |
The :doc:`balancer </core/sharding-balancing>` will redistribute
chunks of documents when it next runs. As clients insert additional
documents into this collection, the :program:`mongos` will distribute the
documents between the shards.
---
title:
text: Confirm the shard is balancing.
character: "`"
stepnum: 4
ref: enable-sharding-confirm
action:
- pre: |
You can confirm balancing activity by switching to the ``test`` database
and running :method:`db.stats()` or :method:`db.printShardingStatus()`.

In the :program:`mongo` shell, issue the following commands to return
statistics for each cluster:
language: sh
code: |
use test
db.stats()
db.printShardingStatus()
- pre: |
Example output of the :method:`db.stats()` command:
language: sh
code: |
{
"raw" : {
"rs0/mongodb0.example.net:27017,mongodb1.example.net:27017,mongodb2.example.net:27017" : {
"db" : "test",
"collections" : 3,
"objects" : 973887,
"avgObjSize" : 100.33173458522396,
"dataSize" : 97711772,
"storageSize" : 141258752,
"numExtents" : 15,
"indexes" : 2,
"indexSize" : 56978544,
"fileSize" : 1006632960,
"nsSizeMB" : 16,
"ok" : 1
},
"rs1/mongodb3.example.net:27017,mongodb4.example.net:27017,mongodb5.example.net:27017" : {
"db" : "test",
"collections" : 3,
"objects" : 26125,
"avgObjSize" : 100.33286124401914,
"dataSize" : 2621196,
"storageSize" : 11194368,
"numExtents" : 8,
"indexes" : 2,
"indexSize" : 2093056,
"fileSize" : 201326592,
"nsSizeMB" : 16,
"ok" : 1
}
},
"objects" : 1000012,
"avgObjSize" : 100.33176401883178,
"dataSize" : 100332968,
"storageSize" : 152453120,
"numExtents" : 23,
"indexes" : 4,
"indexSize" : 59071600,
"fileSize" : 1207959552,
"ok" : 1
}
- pre: |
Example output of the :method:`db.printShardingStatus()` command:
language: sh
code: |
--- Sharding Status ---
sharding version: { "_id" : 1, "version" : 3 }
shards:
{ "_id" : "rs0", "host" : "rs0/mongodb0.example.net:27017,mongodb1.example.net:27017,mongodb2.example.net:27017" }
{ "_id" : "rs1", "host" : "rs1/mongodb3.example.net:27017,mongodb4.example.net:27017,mongodb5.example.net:27017" }
databases:
{ "_id" : "admin", "partitioned" : false, "primary" : "config" }
{ "_id" : "test", "partitioned" : true, "primary" : "rs0" }
test.test_collection chunks:
rs1 5
rs0 186

[...]
post: |
Run these commands for a second time to demonstrate that :term:`chunks
<chunk>` are migrating from ``rs0`` to ``rs1``.
...
40 changes: 40 additions & 0 deletions source/includes/steps-convert-replica-set-shard-deploy-infrastructure.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
title:
text: Start three config databases.
character: "`"
stepnum: 1
ref: start-config-instances
action:
- pre: |
Type this command on each ``mongodb7.example.net``,
``mongodb8.example.net``, and ``mongodb9.example.net`` server:
language: sh
code: |
mongod --configsvr
---
title:
text: Start a mongos instance.
character: "`"
stepnum: 2
ref: start-mongos-instance
pre: |
Start and configure the :program:`mongos` with the location of the
three config servers. Type this command and replace
``mongodb7.example.net``, ``mongodb8.example.net``, and
``mongodb9.example.net`` with your resolvable domains, hostnames, or
IP addresses for your system.
action:
language: sh
code: |
mongos --configdb mongodb07.example.net:27019,mongodb08.example.net:27019,mongodb09.example.net:27019 --chunkSize 1
post: |
If you use the collection created earlier in this tutorial, or
experiment with sharding, you can use a small ``--chunkSize`` (1MB
works well.) The default chunkSize of 64MB means that your cluster
must have 64MB of data before automatic sharding begins working.

In production environments, do not use a small shard size.

The --configdb option specifies the config databases (e.g.
mongodb07.example.net:27019) with their default port number 27019.
The :program:`mongos` runs on the default port 27017.
...
65 changes: 65 additions & 0 deletions source/includes/steps-convert-replica-set-shard-deploy-test-data.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
stepnum: 1
inherit:
file: steps-deploy-replica-set.yaml
ref: start-mongod
post: |
In production deployments, you can configure a :term:`control script` to
manage this process. Control scripts are beyond the scope of this document.

Repeat this step for each member of the replica set.
---
stepnum: 2
inherit:
file: steps-deploy-replica-set.yaml
ref: open-shell
pre: |
Run this command on one member of the replica set (e.g. ``mongodb0.example.net``):
---
stepnum: 3
inherit:
file: steps-deploy-replica-set.yaml
ref: initiate-rs
pre: |
Use :method:`rs.initiate()` on one member of the replica set (e.g. ``mongodb0.example.net``):
---
stepnum: 4
inherit:
file: steps-deploy-replica-set.yaml
ref: verify-rsconf
---
stepnum: 5
inherit:
file: steps-deploy-replica-set.yaml
ref: add-remaining-members
---
title:
text: Create and populate a new collection.
character: "`"
stepnum: 6
ref: create-populate-collection
action:
- pre: |
In the :program:`mongo` shell, issue the following sequence of
JavaScript operations on the primary member of one replica set:
language: javascript
code: |
use test
people = ["Marc", "Bill", "George", "Eliot", "Matt", "Trey", "Tracy", "Greg", "Steve", "Kristina", "Katie", "Jeff"];
for(var i=0; i<1000000; i++){
name = people[Math.floor(Math.random()*people.length)];
user_id = i;
boolean = [true, false][Math.floor(Math.random()*2)];
added_at = new Date();
number = Math.floor(Math.random()*10001);
db.test_collection.save({"name":name, "user_id":user_id, "boolean": boolean, "added_at":added_at, "number":number });
}
- pre: |
The above operations add one million documents to the collection
``test_collection``. This can take several minutes, depending on
your system.

The script adds the documents in the following form:
language: javascript
code: |
{ "_id" : ObjectId("4ed5420b8fc1dd1df5886f70"), "name" : "Greg", "user_id" : 4, "boolean" : true, "added_at" : ISODate("2011-11-29T20:35:23.121Z"), "number" : 74 }
...
37 changes: 26 additions & 11 deletions source/includes/steps-deploy-replica-set.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
title: Start each member of the replica set with the appropriate options.
# This file ref'd in steps-convert-replica-set-shard-deploy-test-data.yaml
# This file is tweaked to work with the other step file, as well.
#
title:
text: Start each member of the replica set with the appropriate options.
character: "`"
stepnum: 1
ref: start-mongod
pre: |
Expand All @@ -24,31 +29,37 @@ post: |
In production deployments, you can configure a :term:`control script` to
manage this process. Control scripts are beyond the scope of this document.
---
title: Open a :program:`mongo` shell and connect to one of the replica set members.
title:
text: Open a :program:`mongo` shell and connect to a replica set member.
character: "`"
stepnum: 2
ref: open-shell
pre: |
For example, to connect to a :program:`mongod` running on localhost on
the default port of ``27017``, simply issue:
action:
pre: |
For example, to connect to a :program:`mongod` running on localhost on
the default port of ``27017``, simply issue:
language: javascript
code: |
mongo
---
title: Initiate the replica set.
title:
text: Initiate the replica set.
character: "`"
stepnum: 3
ref: initiate-rs
pre: |
Use :method:`rs.initiate()` on the replica set member:
action:
pre: |
Use :method:`rs.initiate()`:
language: javascript
code: |
rs.initiate()
post: |
MongoDB initiates a set that consists of the current member and that
uses the default replica set configuration.
---
title: Verify the initial replica set configuration.
title:
text: Verify the initial replica set configuration.
character: "`"
stepnum: 4
ref: verify-rsconf
action:
Expand All @@ -73,7 +84,9 @@ action:
]
}
---
title: Add the remaining members to the replica set.
title:
text: Add the remaining members to the replica set.
character: "`"
stepnum: 5
ref: add-remaining-members
pre: |
Expand All @@ -89,7 +102,9 @@ post: |
When complete, you have a fully functional replica set. The new replica
set will elect a :term:`primary`.
---
title: Check the status of the replica set.
title:
text: Check the status of the replica set.
character: "`"
stepnum: 6
ref: check-status
action:
Expand Down
Loading