Skip to content

Commit e5bfa2b

Browse files
jvincent-mongodblmkerbey-mdb
jvincent-mongodb
and
lmkerbey-mdb
authored
DOCSP-43491 -- Support easy storage (PVC) resize (#1841)
* DOCSP-43491 -- WIP * DOCSP-43491 -- WIP * DOCSP-43491 -- WIP * DOCSP-43491 -- add kubectl describe for PVC resize validation * DOCSP-43491 -- add kubectl describe for PVC resize validation * DOCSP-43491 -- add kubectl describe for PVC resize validation * DOCSP-43491 -- add kubectl describe for PVC resize validation * DOCSP-43491 -- draft + add table * DOCSP-43491 -- draft + add table * DOCSP-43491 -- draft + add table * DOCSP-43491 -- draft + add table * DOCSP-43491 -- draft + add table * DOCSP-43491 -- add tab titles * DOCSP-43491 -- add tab titles * DOCSP-43491 -- add tab titles * DOCSP-43491 -- add easy kubectl steps + manual kubectl steps * DOCSP-43491 -- add easy kubectl steps + manual kubectl steps * DOCSP-43491 -- edit intro paragraph * DOCSP-43491 -- remove Helm tab from manual resize section * DOCSP-43491 -- add RBAC note for easy resize * DOCSP-43491 -- WIP * DOCSP-43491 -- move easy process above manual process * DOCSP-43491 -- copy review revisions * DOCSP-43491 -- WIP external review revisions * DOCSP-43491 -- copy review revisions * DOCSP-43491 -- external review revisions * DOCSP-43491 -- external review revision * DOCSP-43491 -- external review revision * DOCSP-43491 -- external review revisions * Update source/includes/steps-resize-storage-easy.yaml Co-authored-by: lmkerbey-mdb <[email protected]> * Update source/includes/steps-resize-storage-easy.yaml Co-authored-by: lmkerbey-mdb <[email protected]> * Update source/includes/steps-resize-storage-easy.yaml Co-authored-by: lmkerbey-mdb <[email protected]> * Update source/includes/steps-resize-storage.yaml Co-authored-by: lmkerbey-mdb <[email protected]> * Update source/includes/steps-resize-storage.yaml Co-authored-by: lmkerbey-mdb <[email protected]> --------- Co-authored-by: lmkerbey-mdb <[email protected]>
1 parent 2c2c8ef commit e5bfa2b

File tree

3 files changed

+259
-11
lines changed

3 files changed

+259
-11
lines changed

‎source/includes/steps-resize-storage-easy.yaml

Lines changed: 167 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,167 @@
1+
---
2+
stepnum: 1
3+
level: 4
4+
ref: select-resource
5+
title: Create or identify a persistent |k8s-custom-resource|.
6+
content: |
7+
8+
Use an existing database resource or create a new one with persistent
9+
storage. Wait until the persistent volume enters the ``Running``
10+
state.
11+
12+
.. example::
13+
14+
A database resource with persistent storage would include:
15+
16+
.. code-block:: yaml
17+
:linenos:
18+
:emphasize-lines: 11-14
19+
20+
apiVersion: mongodb.com/v1
21+
kind: MongoDB
22+
metadata:
23+
name: <my-replica-set>
24+
spec:
25+
members: 3
26+
version: "4.4.0"
27+
project: my-project
28+
credentials: my-credentials
29+
type: ReplicaSet
30+
podSpec:
31+
persistence:
32+
single:
33+
storage: "1Gi"
34+
35+
---
36+
stepnum: 2
37+
level: 4
38+
ref: insert-data
39+
title: Insert data into the database that the resource serves.
40+
content: |
41+
42+
a. Start |mongo| in the |k8s| cluster.
43+
44+
.. code-block:: sh
45+
46+
$kubectl exec -it <my-replica-set>-0 \
47+
/var/lib/mongodb-mms-automation/mongodb-linux-x86_64-4.4.0/bin/mongo
48+
49+
b. Insert data into the ``test`` database.
50+
51+
.. code-block:: javascript
52+
:copyable: false
53+
54+
<my-replica-set>:PRIMARY> use test
55+
56+
switched to db test
57+
58+
<my-replica-set>:PRIMARY> db.tmp.insertOne({"foo":"bar"})
59+
60+
{
61+
"acknowledged" : true,
62+
"insertedId" : ObjectId("61128cb4a783c3c57ae5142d")
63+
}
64+
65+
---
66+
stepnum: 3
67+
level: 4
68+
ref: update-crd
69+
title: Update the database resource with a new storage value.
70+
content: |
71+
72+
.. important::
73+
74+
You can only increase disk size for existing storage resources, not decrease.
75+
Decreasing the storage size causes an error in the reconcile stage.
76+
77+
a. Update the disk size. Open your preferred text editor and make
78+
changes similar to this example:
79+
80+
.. example::
81+
82+
To update the disk size of the replica set to 2 GB, change the
83+
``storage`` value in the database resource specification:
84+
85+
.. code-block:: yaml
86+
:linenos:
87+
:emphasize-lines: 14
88+
89+
apiVersion: mongodb.com/v1
90+
kind: MongoDB
91+
metadata:
92+
name: <my-replica-set>
93+
spec:
94+
members: 3
95+
version: "4.4.0"
96+
project: my-project
97+
credentials: my-credentials
98+
type: ReplicaSet
99+
podSpec:
100+
persistence:
101+
single:
102+
storage: "2Gi"
103+
104+
b. Update the MongoDB custom resource with the new volume size.
105+
106+
.. code-block:: sh
107+
108+
kubectl apply -f my-updated-replica-set-vol.yaml
109+
110+
c. Wait until this StatefulSet achieves the ``Running`` state.
111+
112+
---
113+
stepnum: 4
114+
level: 4
115+
ref: validate-pv-resize
116+
title: Validate that the |k8s-pvc| has been resized.
117+
content: |
118+
119+
If you reuse |k8s-pvs|, you can find the data that you inserted in **Step
120+
2** on the databases stored in |k8s-pvs|:
121+
122+
.. code-block:: sh
123+
124+
$ kubectl describe mongodb/<my-replica-set> -n mongodb
125+
126+
The following output indicates that your PVC resize request is being processed.
127+
128+
.. code-block:: sh
129+
130+
status:
131+
clusterStatusList: {}
132+
lastTransition: "2024-08-21T11:03:52+02:00"
133+
message: StatefulSet not ready
134+
observedGeneration: 2
135+
phase: Pending
136+
pvc:
137+
- phase: PVC Resize - STS has been orphaned
138+
statefulsetName: multi-replica-set-pvc-resize-0
139+
resourcesNotReady:
140+
- kind: StatefulSet
141+
message: 'Not all the Pods are ready (wanted: 2, updated: 1, ready: 1, current:2)'
142+
name: multi-replica-set-pvc-resize-0
143+
version: ""
144+
145+
---
146+
stepnum: 5
147+
level: 4
148+
ref: validate-pv-change
149+
title: Validate data exists on the updated |k8s-pvc|.
150+
content: |
151+
152+
If you reuse |k8s-pvs|, you can find the data that you inserted in **Step
153+
2** on the databases stored in |k8s-pvs|:
154+
155+
.. code-block:: sh
156+
157+
$ kubectl exec -it <my-replica-set>-1 \
158+
/var/lib/mongodb-mms-automation/mongodb-linux-x86_64-4.4.0/bin/mongo
159+
160+
.. code-block:: javascript
161+
:copyable: false
162+
163+
<my-replica-set>:PRIMARY> use test
164+
switched to db test
165+
166+
<my-replica-set>:PRIMARY> db.tmp.count()
167+
1

‎source/includes/steps-resize-storage.yaml

Lines changed: 30 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -87,10 +87,35 @@ content: |
8787
system resize of volume on node.
8888
status: "True"
8989
type: FileSystemResizePending
90+
9091
---
9192
stepnum: 4
9293
level: 4
9394
ref: recreate-statefulsets
95+
title: Stop the Operator.
96+
content: |
97+
98+
Update the |k8s-op-short| deployment definition and apply the change to your
99+
|k8s| cluster in order to scale the |k8s-op-short| down to ``0`` replicas.
100+
Scaling the |k8s-op-short| down to ``0`` replicas allows you to avoid a race
101+
condition in which the |k8s-op-short| tries to restore the state of the
102+
manually updated resource to align with the resource's original definition.
103+
104+
.. code-block:: yaml
105+
106+
# Source: enterprise-operator/templates/operator.yaml
107+
apiVersion: apps/v1
108+
kind: Deployment
109+
metadata:
110+
name: mongodb-enterprise-operator
111+
namespace: mongodb
112+
spec:
113+
replicas: 0
114+
115+
---
116+
stepnum: 5
117+
level: 4
118+
ref: recreate-statefulsets
94119
title: Remove the StatefulSets.
95120
content: |
96121
@@ -106,7 +131,7 @@ content: |
106131
kubectl delete sts --cascade=false <my-replica-set>
107132
108133
---
109-
stepnum: 5
134+
stepnum: 6
110135
level: 4
111136
ref: update-crd
112137
title: Update the database resource with a new storage value.
@@ -118,7 +143,7 @@ content: |
118143
.. example::
119144
120145
To update the disk size of the replica set to 2 GB, change the
121-
``storage`` value in database resource specification:
146+
``storage`` value in the database resource specification:
122147
123148
.. code-block:: yaml
124149
:linenos:
@@ -145,10 +170,10 @@ content: |
145170
146171
kubectl apply -f my-replica-set-vol.yaml
147172
148-
c. Wait until this StatefulSet achieves the ``Running`` state.
173+
c. Wait until the MongoDB custom resource is in a ``Running`` state.
149174
150175
---
151-
stepnum: 6
176+
stepnum: 7
152177
level: 4
153178
ref: update-pods
154179
title: Update the pods in a rolling fashion.
@@ -163,7 +188,7 @@ content: |
163188
The new pods mount the resized volume.
164189
165190
---
166-
stepnum: 7
191+
stepnum: 8
167192
level: 4
168193
ref: validate-pv-change
169194
title: Validate data exists on the updated |k8s-pvc|.

‎source/tutorial/resize-pv-storage.txt

Lines changed: 62 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
.. _k8s-resize-storage:
22

3-
========================================
4-
Resize Storage for One Database Resource
5-
========================================
3+
=======================================
4+
Increase Storage for Persistent Volumes
5+
=======================================
66

77
.. default-domain:: mongodb
88

@@ -12,6 +12,45 @@ Resize Storage for One Database Resource
1212
:depth: 1
1313
:class: singlecol
1414

15+
The |mms|, MongoDB Database, AppDB and Backup Daemon custom resources that
16+
comprise a standard |k8s-op-short| deployment are each deployed as |k8s|
17+
``statefulSets``. The |k8s-op-short| supports increasing the storage associated with
18+
these specific resources by increasing the capacity of their respective |k8s|
19+
``persistentVolumeClaims`` when the underlying |k8s| ``storageClass`` supports |k8s|
20+
``persistentVolume`` expansion.
21+
22+
Depending on the specific resource type, you can increase storage in one of two ways.
23+
You can either manually increase storage, or you can leverage the |k8s-op-short|
24+
easy storage expansion feature. The following table illustrates which of these two
25+
procedures is supported for a given custom resource type.
26+
27+
.. list-table::
28+
:widths: 40 30 30
29+
:header-rows: 1
30+
31+
* - Custom Resource Type
32+
- Manual Storage Expansion
33+
- Easy Storage Expansion
34+
35+
* - AppDB
36+
- X
37+
-
38+
39+
* - Backup Daemon
40+
- X
41+
-
42+
43+
* - MongoDB Database
44+
- X
45+
- X
46+
47+
* - MongoDB Multi-Cluster
48+
- X
49+
- X
50+
51+
* - Ops Manager
52+
- X
53+
-
1554

1655
Prerequisites
1756
-------------
@@ -27,10 +66,27 @@ use supports resize:
2766
kubectl patch storageclass/<my-storageclass> --type='json' \
2867
-p='[{"op": "add", "path": "/allowVolumeExpansion", "value": true }]'
2968

30-
If you don't have a StorageClass that supports resizing, ask your |k8s|
69+
If you don't have a ``storageClass`` that supports resizing, ask your |k8s|
3170
administrator to help.
3271

33-
Procedure
34-
---------
72+
Easy Expand Storage
73+
-------------------
74+
75+
.. note::
76+
77+
The easy expansion mechanism requires the default RBAC included with the |k8s-op-short|.
78+
Specifically, it requires ``get``, ``list``, ``watch``, ``patch`` and ``update``
79+
permissions for ``persistantVolumeClaims``. If you have customized any of
80+
the |k8s-op-short| RBAC resources, you might need to adjust permissions to
81+
allow the |k8s-op-short| to resize storage resources in your |k8s| cluster.
82+
83+
This process results in a rolling restart of the MongoDB custom resource
84+
in your |k8s| cluster.
85+
86+
.. include:: /includes/steps/resize-storage-easy.rst
87+
88+
Manually Expand Storage
89+
-----------------------
3590

91+
3692
.. include:: /includes/steps/resize-storage.rst

0 commit comments

Comments
 (0)