DOCS-12014 document __rest

Author: Isabella Siu

source/includes/options-mongod.yaml

@@ -39,6 +39,42 @@ description: |
 optional: true
 ---
 program: mongod
+name: configExpand
+args: <none|rest>
+default: |
+  none
+directive: option
+description: |
+  .. versionadded:: 4.2
+
+  Enables using :ref:`Expansion Directives <externally-sourced-values>` to set
+  externally sourced values for configuration file options.
+
+  .. list-table::
+     :header-rows: 1
+     :widths: 20 40
+
+     * - Value
+
+       - Description
+
+     * - ``none``
+
+       - Default. {{program}} does not expand expansion directives. {{program}}
+         fails to start if any configuration file settings use expansion
+         directives.
+
+     * - ``rest``
+
+       - {{program}} only expands ``__rest`` expansion directives. {{program}}
+         fails to start if any configuration file settings use any other
+         expansion directive.
+
+  See :ref:`externally-sourced-values` for configuration files
+  for more information on expansion directives.
+optional: true
+---
+program: mongod
 name: verbose
 inherit:
   name: verbose
@@ -3271,10 +3307,10 @@ description: |
    .. versionadded:: 4.2
 
    Outputs the resolved YAML configuration document for the {{program}} to ``stdout``
-   and halts the server instance. For configuration options using externally
-   sourced values, {{role}} returns the resolved value for those options. This
-   may include any configured passwords or secrets previously obfuscated through
-   the external source.
+   and halts the server instance. For configuration options using 
+   :ref:`externally-sourced-values`, {{role}} returns the resolved value for those
+   options. This may include any configured passwords or secrets previously 
+   obfuscated through the external source.
 
 optional: true
 ---

source/includes/options-mongos.yaml

@@ -27,6 +27,13 @@ inherit:
   file: options-mongod.yaml
 ---
 program: mongos
+name: configExpand
+inherit:
+  name: configExpand
+  program: mongod
+  file: options-mongod.yaml
+---
+program: mongos
 name: verbose
 inherit:
   name: verbose

source/reference/configuration-options.txt

@@ -84,6 +84,99 @@ configuration file, :binary:`~bin.mongod` may not start.
 
 .. [#yaml-json] YAML is a superset of :term:`JSON`.
 
+.. _externally-sourced-values:
+
+Externally Sourced Values
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 4.2
+
+Starting in MongoDB 4.2, users can use expansion directives in the
+configuration file to define the value of configuration file options with
+values from an external source. The following expansion directives are
+available:
+
+- :configexpansion:`__rest`
+
+You must use the :option:`--configExpand <mongod --configExpand>`
+option for the :binary:`~bin.mongod`/:binary:`~bin.mongos` to enable parsing of
+expansion directives. If you specify any expansion directives in the
+configuration file and start the :binary:`~bin.mongod`/:binary:`~bin.mongos`
+without this option, the :binary:`~bin.mongod`/:binary:`~bin.mongos` fails
+to start. This option must be configured on the command line only, and can not
+be expressed in either YAML or INI configuration files.
+ 
+.. configexpansion:: __rest
+
+   The ``__rest`` expansion directive allows users to use externally sourced
+   values in a configuration file. ``__rest`` makes http(s) calls for values
+   and can be used two ways.
+
+   ``__rest`` can fetch field values:
+
+   .. code-block:: yaml
+
+      <field> : 
+         __rest: <string>
+         type: string
+         trim: < none | whitespace >
+
+   ``__rest`` can also be used to fetch entire configuration files:
+
+   .. code-block:: yaml
+
+      __rest: <string>
+      type: yaml
+      trim: < none | whitespace >  
+
+   The ``__rest`` expansion directive takes a string representing the url for
+   which the mongod or mongos issues a ``GET`` request to retrieve the value.
+
+   .. note::
+
+      - ``__rest`` URLs must be HTTPS unless they are referring to
+        localhost, in which case they can use unencrypted HTTP URLs.
+      - If the REST endpoint requires authentication, encode credentials into the
+        URL with standard
+        `RFC 3986 User Information <https://tools.ietf.org/html/rfc3986#section-3.2.1>`_
+        format.
+      - HTTPS requests require TLS 1.1 or later on the client and the host.
+
+   The following optional fields affect the behavior of the ``__rest``
+   expansion directive:
+
+   .. list-table::
+      :header-rows: 1
+      :widths: 20 80
+
+      * - Field
+        - Description
+
+      * - ``type``
+ 
+        - Optional. Defaults to ``string``
+
+          For ``string``, the :binary:`~bin.mongod`/:binary:`~bin.mongos` uses
+          the value returned from the endpoint as a literal string. This value
+          is not parsed for additional structure beyond what is specific to the
+          setting.
+
+          For ``yaml``, the ``__rest`` directive must be the *only* block in
+          the entire configuration file. Users must define their entire
+          configuration file within the specified endpoint, and the return type
+          must be YAML. See the :ref:`yaml example<rest-yaml-example>`.
+
+          .. warning::
+             Recursive ``__rest`` expansion directives are forbidden. If the
+             value returned from a ``__rest`` expansion directive contains an
+             expansion directive, the :binary:`~bin.mongod`/
+             :binary:`~bin.mongos` throws an error and fails to start.
+
+      * - ``trim``
+
+        - Optional. Defaults to ``none``. Specify ``whitespace`` to trim any
+          leading or trailing whitespace from the resulting response.
+
 Use the Configuration File
 ~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -114,6 +207,121 @@ If you installed from a package and have started MongoDB using your
 system's :term:`init script`, you are already using a configuration
 file.
 
+.. _rest-yaml-example:
+
+``__rest`` Expansion with YAML Example
+``````````````````````````````````````
+
+When using expansion directives, you must include the :option:`--configExpand
+<mongod --configExpand>` command line option when starting the the
+:binary:`~bin.mongod`/:binary:`~bin.mongos`.
+
+For example, the configuration file ``mongod.conf`` consists of the following
+``__rest`` expansion directive. 
+
+.. code-block:: yaml
+
+  __rest: "https://mongoconf.example.net:8080/record/1"
+  type: yaml
+
+.. note::
+   When using the ``__rest`` expansion directive with ``type : yaml``, the
+   configuration file must include only the expansion directive and its
+   related options. All other required configuration options must exist on
+   the remote resource specified in ``__rest``.
+
+The YAML document recorded at the URL is:
+
+.. code-block:: yaml
+
+  net:
+     port: 20128
+
+To start :binary:`~bin.mongod` using the configuration, include both the
+configuration file and the :option:`--configExpand <mongod --configExpand>`
+options:
+
+.. code-block:: yaml
+
+  mongod -f mongod.conf --configExpand rest
+
+To verify the configuration, you can include the 
+:option:`--outputConfig <mongod --outputConfig>` option.
+The :option:`--outputConfig <mongod --outputConfig>` option prints the
+specified configuration, including any resolved values,  to ``stdout`` and
+halts the :binary:`~bin.mongod`.
+
+.. warning::
+   
+   The :option:`--outputConfig <mongod --outputConfig>` option returns the
+   resolved values for any field using an expansion directive. This includes
+   any private or sensitive information previously obscured by using an
+   external source for configuration.
+
+For example:
+
+.. code-block:: yaml
+
+   mongod -f mongod.conf --configExpand rest  --outputConfig
+
+Will output:
+
+.. code-block:: yaml
+
+  config: mongod.conf
+  configExpand: rest
+  net:
+    port: 20128
+  outputConfig: true
+
+``--outputConfig`` with Configuration Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The :option:`--outputConfig <mongod --outputConfig>` option causes the
+:binary:`~bin.mongod` or :binary:`~bin.mongos` to output its resolved YAML
+configuration document to ``stdout`` and halt. For configuration options using
+externally sourced values, :option:`--outputConfig <mongod --outputConfig>`
+returns the resolved value for those options.
+
+.. warning::
+   
+   The :option:`--outputConfig <mongod --outputConfig>` option returns the
+   resolved values for any field using an expansion directive. This includes
+   any private or sensitive information previously obscured by using an
+   external source for configuration.
+
+.. example::
+
+   The following configuration file ``mongod.conf`` contains a ``__rest``
+   expansion directive. 
+
+   .. code-block:: yaml
+
+      net:
+         port:
+            __rest: "https://mongoconf.example.net:8080/record/1"
+
+   The string recorded at the URL is:
+
+   .. code-block:: yaml
+
+      20128
+
+   After starting the :binary:`~bin.mongod` with:
+
+   .. code-block:: yaml
+
+      mongod -f mongod.conf --configExpand rest --outputConfig
+
+   The output is:
+
+   .. code-block:: yaml
+
+      config: mongod.conf
+      net:
+        port: 20128
+      outputConfig: true
+
 Core Options
 ------------
 

source/reference/program/mongod.txt

@@ -53,6 +53,9 @@ Core Options
 
 .. include:: /includes/option/option-mongod-config.rst
 
+.. _mongod-configExpand:
+.. include:: /includes/option/option-mongod-configExpand.rst
+
 .. include:: /includes/option/option-mongod-verbose.rst
 
 .. include:: /includes/option/option-mongod-quiet.rst

source/reference/program/mongos.txt

@@ -56,6 +56,8 @@ Core Options
 
 .. include:: /includes/option/option-mongos-config.rst
 
+.. include:: /includes/option/option-mongos-configExpand.rst
+
 .. include:: /includes/option/option-mongos-verbose.rst
 
 .. include:: /includes/option/option-mongos-quiet.rst

source/release-notes/4.2.txt

@@ -267,6 +267,14 @@ applies.
 
 .. seealso:: :ref:`4.2-compatibility-enabled`
 
+Externally Sourced Values for Configuration Files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+MongoDB 4.2 allows users to use :ref:`externally-sourced-values` to define the
+value of configuration options in a configuration file. MongoDB supports
+``__rest`` expansion directives for retrieving values as both strings and YAML
+documents.
+
 Changes Affecting Compatibility
 -------------------------------