DRIVERS-1724 Add backgroundThreadIntervalMS pool option to CMAP test specification (#998)

DRIVERS-1724 Add `backgroundThreadIntervalMS` pool option to CMAP test specification

Use the new option in `pool-checkout-no-idle.yml`, `pool-checkout-no-stale.yml`
to disable background thread runs.
Use the new option in `pool-clear-min-size.yml` and `pool-create-min-size-error.yml`
to specify 50 ms delay between runs.

DRIVERS-1724

Author: stIncMale

source/connection-monitoring-and-pooling/connection-monitoring-and-pooling.rst

@@ -9,8 +9,8 @@ Connection Monitoring and Pooling
 :Status: Accepted
 :Type: Standards
 :Minimum Server Version: N/A
-:Last Modified: 2021-04-12
-:Version: 1.5.0
+:Last Modified: 2021-06-02
+:Version: 1.5.1
 
 .. contents::
 
@@ -417,9 +417,9 @@ method MUST immediately return and MUST NOT emit a PoolReadyEvent.
 
    mark pool as "ready"
    emit PoolReadyEvent
-   resume background thread
+   allow background thread to create connections
 
-Note that resuming the background thread after emitting PoolReadyEvent is of the essence,
+Note that the PoolReadyEvent MUST be emitted before the background thread is allowed to resume creating new connections,
 and it must be the case that no observer is able to observe actions of the background thread
 related to creating new connections before observing the PoolReadyEvent event.
 
@@ -736,12 +736,19 @@ A Pool SHOULD have a background Thread that is responsible for
 monitoring the state of all available `Connections <#connection>`_. This background
 thread SHOULD
 
--  Populate `Connections <#connection>`_ to ensure that the pool always satisfies **minPoolSize**
-    - The background thread SHOULD just go back to sleep instead of waiting for
-      pendingConnectionCount to become less than maxConnecting when satisfying
-      minPoolSize.
+-  Populate `Connections <#connection>`_ to ensure that the pool always satisfies minPoolSize.
 -  Remove and close perished available `Connections <#connection>`_.
 
+Conceptually, the aforementioned activities are organized into sequential Background Thread Runs.
+A Run MUST do as much work as readily available and then end instead of waiting for more work.
+For example, instead of waiting for pendingConnectionCount to become less than maxConnecting when satisfying minPoolSize,
+a Run MUST either proceed with the rest of its duties, e.g., closing available perished connections, or end.
+
+The duration of intervals between the end of one Run and the beginning of the next Run is not specified,
+but the
+`Test Format and Runner Specification <https://github.com/mongodb/specifications/tree/master/source/connection-monitoring-and-pooling/tests>`__
+may restrict this duration, or introduce other restrictions to facilitate testing.
+
 withConnection
 ^^^^^^^^^^^^^^
 
@@ -1115,6 +1122,8 @@ Change log
 
 :2021-4-12: Adding in behaviour for load balancer mode.
 
+:2021-06-02: Formalize the behavior of a `Background Thread <#background-thread>`__.
+
 .. Section for links.
 
 .. _Application Errors: /source/server-discovery-and-monitoring/server-discovery-and-monitoring.rst#application-errors

source/connection-monitoring-and-pooling/tests/README.rst

@@ -36,7 +36,20 @@ Unit Test Format:
 
 All Unit Tests have some of the following fields:
 
-- ``poolOptions``: if present, connection pool options to use when creating a pool
+- ``poolOptions``: If present, connection pool options to use when creating a pool;
+  both `standard ConnectionPoolOptions <https://github.com/mongodb/specifications/blob/master/source/connection-monitoring-and-pooling/connection-monitoring-and-pooling.rst#connection-pool-options-1>`__
+  and the following test-specific options are allowed:
+
+  - ``backgroundThreadIntervalMS``: A time interval between the end of a
+    `Background Thread Run <https://github.com/mongodb/specifications/blob/master/source/connection-monitoring-and-pooling/connection-monitoring-and-pooling.rst#background-thread>`__
+    and the beginning of the next Run. If a Connection Pool does not implement a Background Thread, the Test Runner MUST ignore the option.
+    If the option is not specified, an implementation is free to use any value it finds reasonable.
+
+    Possible values (0 is not allowed):
+
+    - A negative value: never begin a Run.
+    - A positive value: the interval between Runs in milliseconds.
+
 - ``operations``: A list of operations to perform. All operations support the following fields:
 
   - ``name``: A string describing which operation to issue.
@@ -155,8 +168,6 @@ For each YAML file with ``style: unit``:
 
   - If ``poolOptions`` is specified, use those options to initialize both pools
   - The returned pool must have an ``address`` set as a string value.
-  - If the pool uses a background thread to satisfy ``minPoolSize``, ensure it
-    attempts to create a new connection every 50ms.
 
 - Process each ``operation`` in ``operations`` (on the main thread)
 

source/connection-monitoring-and-pooling/tests/pool-checkout-no-idle.json

@@ -3,7 +3,8 @@
   "style": "unit",
   "description": "must destroy and must not check out an idle connection if found while iterating available connections",
   "poolOptions": {
-    "maxIdleTimeMS": 10
+    "maxIdleTimeMS": 10,
+    "backgroundThreadIntervalMS": -1
   },
   "operations": [
     {
@@ -24,11 +25,6 @@
     {
       "name": "checkOut"
     },
-    {
-      "name": "waitForEvent",
-      "event": "ConnectionClosed",
-      "count": 1
-    },
     {
       "name": "waitForEvent",
       "event": "ConnectionCheckedOut",
@@ -56,6 +52,11 @@
       "connectionId": 1,
       "reason": "idle",
       "address": 42
+    },
+    {
+      "type": "ConnectionCheckedOut",
+      "connectionId": 2,
+      "address": 42
     }
   ],
   "ignore": [

source/connection-monitoring-and-pooling/tests/pool-checkout-no-idle.yml

@@ -3,6 +3,7 @@ style: unit
 description: must destroy and must not check out an idle connection if found while iterating available connections
 poolOptions:
   maxIdleTimeMS: 10
+  backgroundThreadIntervalMS: -1
 operations:
   - name: ready
   - name: checkOut
@@ -12,9 +13,6 @@ operations:
   - name: wait
     ms: 50
   - name: checkOut
-  - name: waitForEvent
-    event: ConnectionClosed
-    count: 1
   - name: waitForEvent
     event: ConnectionCheckedOut
     count: 2
@@ -33,6 +31,9 @@ events:
     connectionId: 1
     reason: idle
     address: 42
+  - type: ConnectionCheckedOut
+    connectionId: 2
+    address: 42
 ignore:
   - ConnectionReady
   - ConnectionPoolReady

source/connection-monitoring-and-pooling/tests/pool-checkout-no-stale.json

@@ -2,6 +2,9 @@
   "version": 1,
   "style": "unit",
   "description": "must destroy and must not check out a stale connection if found while iterating available connections",
+  "poolOptions": {
+    "backgroundThreadIntervalMS": -1
+  },
   "operations": [
     {
       "name": "ready"
@@ -23,11 +26,6 @@
     {
       "name": "checkOut"
     },
-    {
-      "name": "waitForEvent",
-      "event": "ConnectionClosed",
-      "count": 1
-    },
     {
       "name": "waitForEvent",
       "event": "ConnectionCheckedOut",
@@ -59,6 +57,11 @@
       "connectionId": 1,
       "reason": "stale",
       "address": 42
+    },
+    {
+      "type": "ConnectionCheckedOut",
+      "connectionId": 2,
+      "address": 42
     }
   ],
   "ignore": [

source/connection-monitoring-and-pooling/tests/pool-checkout-no-stale.yml

@@ -1,6 +1,8 @@
 version: 1
 style: unit
 description: must destroy and must not check out a stale connection if found while iterating available connections
+poolOptions:
+  backgroundThreadIntervalMS: -1
 operations:
   - name: ready
   - name: checkOut
@@ -10,9 +12,6 @@ operations:
   - name: clear
   - name: ready
   - name: checkOut
-  - name: waitForEvent
-    event: ConnectionClosed
-    count: 1
   - name: waitForEvent
     event: ConnectionCheckedOut
     count: 2
@@ -32,6 +31,9 @@ events:
     connectionId: 1
     reason: stale
     address: 42
+  - type: ConnectionCheckedOut
+    connectionId: 2
+    address: 42
 ignore:
   - ConnectionReady
   - ConnectionPoolReady

source/connection-monitoring-and-pooling/tests/pool-clear-min-size.json

@@ -3,7 +3,8 @@
   "style": "unit",
   "description": "pool clear halts background minPoolSize establishments",
   "poolOptions": {
-    "minPoolSize": 1
+    "minPoolSize": 1,
+    "backgroundThreadIntervalMS": 50
   },
   "operations": [
     {

source/connection-monitoring-and-pooling/tests/pool-clear-min-size.yml

@@ -3,6 +3,7 @@ style: unit
 description: pool clear halts background minPoolSize establishments
 poolOptions:
   minPoolSize: 1
+  backgroundThreadIntervalMS: 50
 operations:
   - name: ready
   - name: waitForEvent

source/connection-monitoring-and-pooling/tests/pool-create-min-size-error.json

@@ -23,6 +23,7 @@
   },
   "poolOptions": {
     "minPoolSize": 1,
+    "backgroundThreadIntervalMS": 50,
     "appName": "poolCreateMinSizeErrorTest"
   },
   "operations": [

source/connection-monitoring-and-pooling/tests/pool-create-min-size-error.yml

@@ -15,6 +15,7 @@ failPoint:
     appName: "poolCreateMinSizeErrorTest"
 poolOptions:
   minPoolSize: 1
+  backgroundThreadIntervalMS: 50
   appName: "poolCreateMinSizeErrorTest"
 operations:
   - name: ready