Skip to content

Commit 81903fb

Browse files
davidhou17jwilliams-mongo
davidhou17
authored and
jwilliams-mongo
committed
(DOCSP-32924): Add two new params that enable users to fix a broken Automation Config (#1477)
* Add new params for recovery * Adds troubleshooting section and shortens RN * Add example to troubleshooting section
1 parent f78945e commit 81903fb

File tree

5 files changed

+118
-12
lines changed

5 files changed

+118
-12
lines changed

‎source/includes/op-setting-descs/mdb-automatic-backoff-time-s.rst

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
Number of seconds that a custom resource can remain in a
2+
``Pending`` or ``Failed`` state before the |k8s-op-short|
3+
automatically recovers your ``MongoDB`` resources.
4+
5+
The default value
6+
is **1200** seconds (20 minutes).
7+
8+
To disable automatic recovery, set the
9+
:ref:`MDB_AUTOMATIC_RECOVERY_ENABLE <mdb-automatic-recovery-enable>`
10+
environment variable to ``false``.

‎source/includes/op-setting-descs/mdb-automatic-recovery-enable.rst

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
Flag that determines whether or not the |k8s-op-short|
2+
enables automatic recovery for ``MongoDB`` resources
3+
per Pod.
4+
5+
The default value is **true**.

‎source/reference/kubectl-operator-settings.txt

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -374,6 +374,54 @@ The default value is ``false``, which automatically updates the image suffix.
374374
- name: MDB_APPDB_ASSUME_OLD_FORMAT
375375
value: false
376376

377+
.. _mdb-automatic-recovery-enable:
378+
379+
MDB_AUTOMATIC_RECOVERY_ENABLE
380+
-----------------------------
381+
382+
.. include:: /includes/op-setting-descs/mdb-automatic-recovery-enable.rst
383+
384+
.. example::
385+
386+
.. code-block:: yaml
387+
:linenos:
388+
389+
spec:
390+
template:
391+
spec:
392+
serviceAccountName: mongodb-enterprise-operator
393+
containers:
394+
- name: mongodb-enterprise-operator
395+
image: <operatorVersionUrl>
396+
imagePullPolicy: <policyChoice>
397+
env:
398+
- name: MDB_AUTOMATIC_RECOVERY_ENABLE
399+
value: false
400+
401+
.. _mdb-automatic-backoff-time-s:
402+
403+
MDB_AUTOMATIC_RECOVERY_BACKOFF_TIME_S
404+
-------------------------------------
405+
406+
.. include:: /includes/op-setting-descs/mdb-automatic-backoff-time-s.rst
407+
408+
.. example::
409+
410+
.. code-block:: yaml
411+
:linenos:
412+
413+
spec:
414+
template:
415+
spec:
416+
serviceAccountName: mongodb-enterprise-operator
417+
containers:
418+
- name: mongodb-enterprise-operator
419+
image: <operatorVersionUrl>
420+
imagePullPolicy: <policyChoice>
421+
env:
422+
- name: MDB_AUTOMATIC_RECOVERY_BACKOFF_TIME_S
423+
value: 3600
424+
377425
.. _mdb-image-type:
378426

379427
MDB_IMAGE_TYPE

‎source/reference/troubleshooting.txt

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -841,3 +841,52 @@ following command to remove it:
841841
.. code-block:: sh
842842

843843
kubectl delete deployment/enterprise-operator
844+
845+
.. _troubleshooting-automatic-recovery:
846+
847+
Recover Resource Due to Broken Automation Configuration
848+
-------------------------------------------------------
849+
850+
If a custom resource remains in a ``Pending`` or ``Failed`` state
851+
for a longer period of time, |k8s-op-short| automatically recovers
852+
your ``MongoDB`` resources by pushing the :opsmgr:`automation configuration
853+
</reference/cluster-configuration>` to |onprem|. This prevents a deadlock
854+
when the MongoDB Agent can't push an updated automation configuration change
855+
because the StatefulSet is stuck in a ``Pending`` state due to a previous
856+
push of an invalid automation configuration.
857+
858+
To configure automatic recovery, define the following environmental variables
859+
in your :github:`mongodb-enterprise.yaml
860+
</mongodb/mongodb-enterprise-kubernetes/blob/master/mongodb-enterprise.yaml>`
861+
file:
862+
863+
- :ref:`MDB_AUTOMATIC_RECOVERY_ENABLE <mdb-automatic-recovery-enable>`
864+
to enable or disable automatic recovery for ``MongoDB`` resources per Pod.
865+
866+
- :ref:`MDB_AUTOMATIC_RECOVERY_BACKOFF_TIME_S <mdb-automatic-backoff-time-s>`
867+
to set the number of seconds that a custom resource can remain in a ``Pending`` or
868+
``Failed`` state before the |k8s-op-short| automatically recovers your
869+
``MongoDB`` resources.
870+
871+
.. example::
872+
873+
.. code-block:: yaml
874+
:linenos:
875+
876+
spec:
877+
template:
878+
spec:
879+
serviceAccountName: mongodb-enterprise-operator
880+
containers:
881+
- name: mongodb-enterprise-operator
882+
image: <operatorVersionUrl>
883+
imagePullPolicy: <policyChoice>
884+
env:
885+
- name: MDB_AUTOMATIC_RECOVERY_ENABLE
886+
value: true
887+
- name: MDB_AUTOMATIC_RECOVERY_BACKOFF_TIME_S
888+
value: 1200
889+
890+
To learn how to define environment variables, see
891+
:k8sdocs:`Define Environment Variables for a Container
892+
</tasks/inject-data-application/define-environment-variable-container>`.

‎source/release-notes.txt

Lines changed: 6 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -82,20 +82,14 @@ Improvements
8282
++++++++++++
8383

8484
- Adds an automatic recovery mechanism for ``MongoDB``
85-
resources. If a custom resource remains in ``Pending`` or ``Failed`` state
86-
for a longer period of time, the automation configuration is pushed to
87-
|onprem|. This prevents a deadlock when the MongoDB Agent can't push an
88-
updated automation configuration change because the StatefulSet is stuck
89-
in a ``Pending`` state due to a previous push of an invalid automation configuration.
85+
resources when a custom resource remains in a ``Pending``
86+
or ``Failed`` state for a longer period of time. In addition,
87+
introduces the following environment variables:
9088

91-
In addition, introduces the following environment variables:
89+
- :ref:`MDB_AUTOMATIC_RECOVERY_ENABLE <mdb-automatic-recovery-enable>`
90+
- :ref:`MDB_AUTOMATIC_RECOVERY_BACKOFF_TIME_S <mdb-automatic-backoff-time-s>`
9291

93-
- ``MDB_AUTOMATIC_RECOVERY_ENABLE`` controls automatic recovery for
94-
``MongoDB`` resources per Pod. The default is ``true``.
95-
96-
- ``MDB_AUTOMATIC_RECOVERY_BACKOFF_TIME_S`` controls the time a custom
97-
resource remains in a ``Pending`` or ``Failed`` state. Set this variable
98-
at the |k8s-op-short| Pod spec level. The default is 1200 seconds (20 minutes).
92+
To learn more, see :ref:`troubleshooting-automatic-recovery`.
9993

10094
- Allows you to route the audit logs for the ``MongoDB`` resource to the |k8s|
10195
Pod logs. Ensure that you write the ``MongoDB`` resource's audit logs

0 commit comments

Comments
 (0)