Skip to content

Commit ded9ae5

Browse files
CDRIVER-4114: Reduce contention on the topology description (#854)
* Additional atomic operations * 64bit and pointer atomics for MSVC * Revive some old exported symbols, clean up some unused code, and mark unreachable code. * No 64bit atomics for x86 GCC either. Fix several warnings * Token balance in case of empty macro expansion * Fix extra semicolon, plus macOS posix detection * Fix check on 32-bit, and missing 'default' case. Closes: CDRIVER-4124 * Missing ptr_fetch * CXX-4113: Thread-safe pool for server session objects * Remove unused node pointers on server sessions * Track pool size, and don't loop while holding the lock to clear * Closes: CDRIVER-4124 * Wrong arg names in i64 compare_exchange emul * Missing dist listing for ts-pool files * Cleanup and some redesign of pool API * Declare special pool for server sessions * Fix pruning pool not leaking the 10,001st session * Use typed pool fns * Visiting pool items, and checking that items are returned to the pool * Fix unacknowledged sessions tests * Additional atomic operations * A simple shared pointer construct * More atomics * Fix atomic intrinsics on MSVC * Topology description is a shared ptr * Give sptr._aux a type to aid in debugging. And no spinlocks! * Fix race on updating server rtt * Assertion macro for asserting pointers inline * Refactoring many locations to copy the shared topology description * Undo accidental local dev changes * Give the topology scanner its own mutex for the handshake_cmd This requires a refactor of how the handshake_cmd is passed around. Instead of the scanner returning a pointer to contended data and expecting the caller to bring their own synchronization, the scanner duplicates the handshake into an output parameter and performs its own synchronization internally. * Instead of using a mutex to guard monitor thread startup, use an atomic compare-exchange. * Pull from new-atomics * Fix: 'release' is not valid for GCC compare-exchange * Lots of "const" weeds out threading a mutation issues * Fix const-init from non-constant-expr * Use BSON_OS_UNIX * fetch() is `const`, and no `release` in cmpxch * Fix missing 'inline's * Mention the LIFO nature of the thread-pool, and the non-conformance of the 'push' operation * auditing the pool is compile-time optional * A shared-pointer abstraction * Make mongoc-shared a private API * typo * init the shared_ptr_mtx in a Windows-compatible way. * Update names to reflect C11 names, strong/weak cmpxchg * Fix const-to-mutable bind in MSVC * PR and impl updates for shared_ptr: - Rename to more closely match stdlib equivs - Delete macros that are unneeded - Tweak impl of atomic_store to be more efficient - Update doc comments * Address various PR comments * More tests for atomic operations * Restore bson_memory_barrier with a deprecation notice * Spelling is hard * No 'consume' memorder for atomic exchange * Fix detection of 64bit, plus unused intrin value * Missing i64_fetch and i64_fetch_sub on 32-bit playforms * Fix spinlock logic inversion for i64 emul * Remove config checks for atomics * Just use `int` * Thread-safe CSE startup without a mutex * Set march=i686 for 32bit builds * Update src/libmongoc/src/mongoc/mongoc-shared.c Co-authored-by: Ezra Chung <[email protected]> * PR comments, use bson_assert_param, protect from self-assignment * CheckAtomics.cmake is gone * PR comments, and an assertion fix * Update doc comments from PR comments * Test aliasing to subobject, and fix C99 usage * Remove C99 usage, unneeded assertions, and warnings * Fix lost topology update with mongoc_topology_reconcile * Refresh td ptr after making changes in a test * Fix deadlocks in topology shutdown * Guard against a zero-allocated mongoc_set * Fix invalid test result JSON generation * Fix warnings from MSVC. Fix unexported symbols used on 32bit DLL * That is not 'const' * Remove C++ comments * Fix leak from misuse of bson_copy_to * Simplify scanner handshake setup * Fix leak of shared pointer when committing topo updates * Fix unsafe use-after free in test case * Missing cast-from-const in int64 atomic emul * Remove unneeded NULL checks. Fix doubled events when racing to invalidate servers * Use generation to validate server description * Fix attempt to double-reconnect * Update _mongoc_cluster_create_server_stream in conditional code segments * Conditionally set error in _cluster_fetch_sream_single * Refresh topology after requesting a scan * Fix race with the scanner to finish the initial handshake * Fix leak of topology descr reference * Fix leaked alloc of bson_t in _stream_run_hello * More leaks with dup_handshake_cmd * BSON initializer that is safe to destroy or to leak. * Undo incorrect changes for bson memcheck code * Less macro magic * Propagate the random seed when cloning a topology description * Prevent recursive topology modifications, which deadlock * Update comments and cleanup. - Remove mentions of topology->mutex. - Update comments that refer to locking. - Remove some redundant APIs. - Move some APIs to have internal linkage. * Guard against a deadlock with an assertion * Revert "Guard against a deadlock with an assertion" This reverts commit 4b9892e. * Update for PR comments. Cleanup, const, and typos * Fix use-after free of server description in test. Remove unneeded locking * Fix decl-after-stmt * Fix shadowed name * Pin docutils version when installing sphinx * Separate mutex for srv polling, document more usage of tpld_modification_mtx * Suppress unused-var warning * ThreadSanitizer calls me out for wrong atomics. * Fix race in on counters in counter test * Release the topology description outside of the mutex * Thread-safe direct topology modification for testing purposes * Fix unnecessary race on getting topology event counts * Thread-safety in tests around topology reconciliation * Fix regression in counters test * Switch sign in counters tests * More thread safety in tests around topology descriptions * Spell * Update to comments, docs, and fixes from PR * Score scanner_state as an int, so we can perform well-defined atomic ops on it * temporarily disable silence in 'run auth tests' * Fix missing close quote in evergreen test script * Temporarily enable logging for 'run auth tests' * Give generation map const-correctness The generation map is owned by its server description, so the pointer member should have deep-const semantics. This uncovers new potential correctness issues, which are fixed now. * Fix TSan warning on server sets when running down * Clean up test usages of unsafe tpld access * Copy the maxElectionId for the topology description * Clear last_hello_response when invalidating server descriptions * Missed modification begin on topology * Remove redundant casts * Re-silence 'run auth tests' Co-authored-by: Ezra Chung <[email protected]>
1 parent 0eeff0a commit ded9ae5

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

56 files changed

+2505
-1921
lines changed

.evergreen/config.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -85,7 +85,7 @@ functions:
8585
python -m virtualenv venv
8686
cd venv
8787
. bin/activate
88-
./bin/pip install sphinx
88+
./bin/pip install sphinx docutils==0.17.1
8989
cd ..
9090
export MONGOC_TEST_FUTURE_TIMEOUT_MS=30000
9191
export MONGOC_TEST_SKIP_LIVE=on
@@ -483,7 +483,7 @@ functions:
483483
export OBSOLETE_TLS='${obsolete_tls}'
484484
export VALGRIND='${valgrind}'
485485
export ATLAS_SERVERLESS_SRV='${atlas_serverless_srv}'
486-
export ATLAS_SERVERLESS='${atlas_serverless}
486+
export ATLAS_SERVERLESS='${atlas_serverless}'
487487
sh .evergreen/run-auth-tests.sh
488488
run mock server tests:
489489
- command: shell.exec

build/evergreen_config_lib/functions.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@
7272
python -m virtualenv venv
7373
cd venv
7474
. bin/activate
75-
./bin/pip install sphinx
75+
./bin/pip install sphinx docutils==0.17.1
7676
cd ..
7777
7878
export MONGOC_TEST_FUTURE_TIMEOUT_MS=30000
@@ -344,7 +344,7 @@
344344
export OBSOLETE_TLS='${obsolete_tls}'
345345
export VALGRIND='${valgrind}'
346346
export ATLAS_SERVERLESS_SRV='${atlas_serverless_srv}'
347-
export ATLAS_SERVERLESS='${atlas_serverless}
347+
export ATLAS_SERVERLESS='${atlas_serverless}'
348348
sh .evergreen/run-auth-tests.sh
349349
''', silent=True),
350350
)),

src/libbson/src/bson/bson-atomic.h

Lines changed: 9 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -299,27 +299,27 @@ DECL_ATOMIC_STDINT (int32, )
299299
DECL_ATOMIC_INTEGRAL (int, int, )
300300
#endif
301301

302-
extern int64_t
302+
BSON_EXPORT (int64_t)
303303
_bson_emul_atomic_int64_fetch_add (int64_t volatile *val,
304304
int64_t v,
305305
enum bson_memory_order);
306-
extern int64_t
306+
BSON_EXPORT (int64_t)
307307
_bson_emul_atomic_int64_exchange (int64_t volatile *val,
308308
int64_t v,
309309
enum bson_memory_order);
310-
extern int64_t
310+
BSON_EXPORT (int64_t)
311311
_bson_emul_atomic_int64_compare_exchange_strong (int64_t volatile *val,
312312
int64_t expect_value,
313313
int64_t new_value,
314314
enum bson_memory_order);
315315

316-
extern int64_t
316+
BSON_EXPORT (int64_t)
317317
_bson_emul_atomic_int64_compare_exchange_weak (int64_t volatile *val,
318318
int64_t expect_value,
319319
int64_t new_value,
320320
enum bson_memory_order);
321321

322-
extern void
322+
BSON_EXPORT (void)
323323
bson_thrd_yield (void);
324324

325325
#if (defined(_MSC_VER) && !defined(_M_IX86)) || (defined(__LP64__) && __LP64__)
@@ -331,9 +331,11 @@ DECL_ATOMIC_STDINT (int64, 64)
331331
#endif
332332
#else
333333
static BSON_INLINE int64_t
334-
bson_atomic_int64_fetch (int64_t volatile *val, enum bson_memory_order order)
334+
bson_atomic_int64_fetch (const int64_t volatile *val,
335+
enum bson_memory_order order)
335336
{
336-
return _bson_emul_atomic_int64_fetch_add (val, 0, order);
337+
return _bson_emul_atomic_int64_fetch_add (
338+
(int64_t volatile *) val, 0, order);
337339
}
338340

339341
static BSON_INLINE int64_t

src/libbson/src/bson/bson-macros.h

Lines changed: 43 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -166,8 +166,8 @@
166166
#else
167167
#define BSON_ALIGNED_BEGIN(_N)
168168
#define BSON_ALIGNED_END(_N) \
169-
__attribute__ ( \
170-
(aligned ((_N) > BSON_ALIGN_OF_PTR ? BSON_ALIGN_OF_PTR : (_N))))
169+
__attribute__ (( \
170+
aligned ((_N) > BSON_ALIGN_OF_PTR ? BSON_ALIGN_OF_PTR : (_N))))
171171
#endif
172172
#endif
173173

@@ -196,18 +196,48 @@
196196
abort (); \
197197
} \
198198
} while (0)
199-
199+
200+
/**
201+
* @brief Assert the expression `Assertion`, and evaluates to `Value` on
202+
* success.
203+
*/
204+
#define BSON_ASSERT_INLINE(Assertion, Value) \
205+
((void) ((Assertion) ? (0) \
206+
: ((fprintf (stderr, \
207+
"%s:%d %s(): Assertion '%s' failed", \
208+
__FILE__, \
209+
__LINE__, \
210+
BSON_FUNC, \
211+
#Assertion), \
212+
abort ()), \
213+
0)), \
214+
Value)
215+
216+
/**
217+
* @brief Assert that the given pointer is non-NULL, while also evaluating to
218+
* that pointer.
219+
*
220+
* Can be used to inline assertions with a pointer dereference:
221+
*
222+
* ```
223+
* foo* f = get_foo();
224+
* bar* b = BSON_ASSERT_PTR_INLINE(f)->bar_value;
225+
* ```
226+
*/
227+
#define BSON_ASSERT_PTR_INLINE(Pointer) \
228+
BSON_ASSERT_INLINE ((Pointer) != NULL, (Pointer))
229+
200230
/* Used for asserting parameters to provide a more precise error message */
201-
#define BSON_ASSERT_PARAM(param) \
202-
do { \
203-
if ((BSON_UNLIKELY (param == NULL))) { \
204-
fprintf (stderr, \
205-
"The parameter: %s, in function %s, cannot be NULL\n", \
206-
#param, \
207-
BSON_FUNC); \
208-
abort (); \
209-
} \
210-
} while (0)
231+
#define BSON_ASSERT_PARAM(param) \
232+
do { \
233+
if ((BSON_UNLIKELY (param == NULL))) { \
234+
fprintf (stderr, \
235+
"The parameter: %s, in function %s, cannot be NULL\n", \
236+
#param, \
237+
BSON_FUNC); \
238+
abort (); \
239+
} \
240+
} while (0)
211241

212242
/* obsolete macros, preserved for compatibility */
213243
#define BSON_STATIC_ASSERT(s) BSON_STATIC_ASSERT_ (s, __LINE__)

src/libbson/src/bson/bson-types.h

Lines changed: 7 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -129,10 +129,10 @@ typedef struct _bson_json_opts_t bson_json_opts_t;
129129
#ifdef BSON_MEMCHECK
130130
BSON_ALIGNED_BEGIN (128)
131131
typedef struct _bson_t {
132-
uint32_t flags; /* Internal flags for the bson_t. */
133-
uint32_t len; /* Length of BSON data. */
134-
char *canary; /* For valgrind check */
135-
uint8_t padding[120 - sizeof (char*)];
132+
uint32_t flags; /* Internal flags for the bson_t. */
133+
uint32_t len; /* Length of BSON data. */
134+
char *canary; /* For valgrind check */
135+
uint8_t padding[120 - sizeof (char *)];
136136
} bson_t BSON_ALIGNED_END (128);
137137
#else
138138
BSON_ALIGNED_BEGIN (128)
@@ -143,7 +143,6 @@ typedef struct _bson_t {
143143
} bson_t BSON_ALIGNED_END (128);
144144
#endif
145145

146-
147146
/**
148147
* BSON_INITIALIZER:
149148
*
@@ -155,13 +154,9 @@ typedef struct _bson_t {
155154
* ]|
156155
*/
157156
#ifdef BSON_MEMCHECK
158-
#define BSON_INITIALIZER \
159-
{ \
160-
3, 5, \
161-
bson_malloc (1), \
162-
{ \
163-
5 \
164-
}, \
157+
#define BSON_INITIALIZER \
158+
{ \
159+
3, 5, bson_malloc (1), {5}, \
165160
}
166161
#else
167162
#define BSON_INITIALIZER \

src/libmongoc/CMakeLists.txt

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -608,7 +608,6 @@ set (HEADERS
608608
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-server-api.h
609609
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-server-description.h
610610
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-client-session.h
611-
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-shared-private.h
612611
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-socket.h
613612
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-stream-tls-libressl.h
614613
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-stream-tls-openssl.h
@@ -618,7 +617,6 @@ set (HEADERS
618617
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-stream-gridfs.h
619618
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-stream-socket.h
620619
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-topology-description.h
621-
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-ts-pool-private.h
622620
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-uri.h
623621
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-version-functions.h
624622
${PROJECT_SOURCE_DIR}/src/mongoc/mongoc-write-concern.h

src/libmongoc/doc/mongoc_topology_description_has_writable_server.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ Synopsis
1010
1111
bool
1212
mongoc_topology_description_has_writable_server (
13-
mongoc_topology_description_t *td);
13+
const mongoc_topology_description_t *td);
1414
1515
Determines if the topology has a writable server available, such as a primary, mongos, or standalone. This function uses the driver's current knowledge of the state of the MongoDB server or servers it is connected to; it does no I/O and it does not block.
1616

src/libmongoc/src/mongoc/mongoc-client-pool.c

Lines changed: 10 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -226,9 +226,7 @@ static void
226226
_start_scanner_if_needed (mongoc_client_pool_t *pool)
227227
{
228228
if (!pool->topology->single_threaded) {
229-
bson_mutex_lock (&pool->topology->mutex);
230229
_mongoc_topology_background_monitoring_start (pool->topology);
231-
bson_mutex_unlock (&pool->topology->mutex);
232230
}
233231
}
234232

@@ -446,30 +444,30 @@ mongoc_client_pool_set_apm_callbacks (mongoc_client_pool_t *pool,
446444
mongoc_apm_callbacks_t *callbacks,
447445
void *context)
448446
{
449-
mongoc_topology_t *topology;
450-
451-
topology = pool->topology;
447+
mongoc_topology_t *const topology = BSON_ASSERT_PTR_INLINE (pool)->topology;
448+
mc_tpld_modification tdmod;
452449

453450
if (pool->apm_callbacks_set) {
454451
MONGOC_ERROR ("Can only set callbacks once");
455452
return false;
456453
}
457454

458-
bson_mutex_lock (&topology->mutex);
455+
tdmod = mc_tpld_modify_begin (topology);
459456

460457
if (callbacks) {
461-
memcpy (&topology->description.apm_callbacks,
458+
memcpy (&tdmod.new_td->apm_callbacks,
462459
callbacks,
463460
sizeof (mongoc_apm_callbacks_t));
464461
memcpy (&pool->apm_callbacks, callbacks, sizeof (mongoc_apm_callbacks_t));
465462
}
466463

467-
mongoc_topology_set_apm_callbacks (topology, callbacks, context);
468-
topology->description.apm_context = context;
464+
mongoc_topology_set_apm_callbacks (
465+
topology, tdmod.new_td, callbacks, context);
466+
tdmod.new_td->apm_context = context;
469467
pool->apm_context = context;
470468
pool->apm_callbacks_set = true;
471469

472-
bson_mutex_unlock (&topology->mutex);
470+
mc_tpld_modify_commit (tdmod);
473471

474472
return true;
475473
}
@@ -540,8 +538,8 @@ mongoc_client_pool_set_server_api (mongoc_client_pool_t *pool,
540538
}
541539

542540
pool->api = mongoc_server_api_copy (api);
543-
bson_mutex_lock (&pool->topology->mutex);
541+
544542
_mongoc_topology_scanner_set_server_api (pool->topology->scanner, api);
545-
bson_mutex_unlock (&pool->topology->mutex);
543+
546544
return true;
547545
}

src/libmongoc/src/mongoc/mongoc-client-side-encryption.c

Lines changed: 42 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1231,14 +1231,14 @@ _mongoc_cse_client_enable_auto_encryption (mongoc_client_t *client,
12311231
GOTO (fail);
12321232
}
12331233

1234-
if (client->topology->cse_enabled) {
1234+
if (client->topology->cse_state != MONGOC_CSE_DISABLED) {
12351235
bson_set_error (error,
12361236
MONGOC_ERROR_CLIENT,
12371237
MONGOC_ERROR_CLIENT_INVALID_ENCRYPTION_STATE,
12381238
"Automatic encryption already set");
12391239
GOTO (fail);
12401240
} else {
1241-
client->topology->cse_enabled = true;
1241+
client->topology->cse_state = MONGOC_CSE_ENABLED;
12421242
}
12431243

12441244
if (!_parse_extra (opts->extra, client->topology, &mongocryptd_uri, error)) {
@@ -1320,11 +1320,11 @@ _mongoc_cse_client_pool_enable_auto_encryption (
13201320
mongoc_auto_encryption_opts_t *opts,
13211321
bson_error_t *error)
13221322
{
1323-
bool ret = false;
1323+
bool setup_okay = false;
13241324
mongoc_uri_t *mongocryptd_uri = NULL;
1325+
mongoc_topology_cse_state_t prev_cse_state = MONGOC_CSE_STARTING;
13251326

13261327
BSON_ASSERT (topology);
1327-
bson_mutex_lock (&topology->mutex);
13281328
if (!opts) {
13291329
bson_set_error (error,
13301330
MONGOC_ERROR_CLIENT,
@@ -1359,16 +1359,32 @@ _mongoc_cse_client_pool_enable_auto_encryption (
13591359
GOTO (fail);
13601360
}
13611361

1362-
if (topology->cse_enabled) {
1362+
prev_cse_state =
1363+
bson_atomic_int_compare_exchange_strong ((int *) &topology->cse_state,
1364+
MONGOC_CSE_DISABLED,
1365+
MONGOC_CSE_STARTING,
1366+
bson_memory_order_acquire);
1367+
while (prev_cse_state == MONGOC_CSE_STARTING) {
1368+
/* Another thread is starting client-side encryption. It may take some
1369+
* time to start, but don't continue until it is finished. */
1370+
bson_thrd_yield ();
1371+
prev_cse_state =
1372+
bson_atomic_int_compare_exchange_strong ((int *) &topology->cse_state,
1373+
MONGOC_CSE_DISABLED,
1374+
MONGOC_CSE_STARTING,
1375+
bson_memory_order_acquire);
1376+
}
1377+
1378+
if (prev_cse_state == MONGOC_CSE_ENABLED) {
13631379
bson_set_error (error,
13641380
MONGOC_ERROR_CLIENT,
13651381
MONGOC_ERROR_CLIENT_INVALID_ENCRYPTION_STATE,
13661382
"Automatic encryption already set");
13671383
GOTO (fail);
1368-
} else {
1369-
topology->cse_enabled = true;
13701384
}
13711385

1386+
/* We just set the CSE state from DISABLED to STARTING. Start it up now. */
1387+
13721388
if (!_parse_extra (opts->extra, topology, &mongocryptd_uri, error)) {
13731389
GOTO (fail);
13741390
}
@@ -1408,11 +1424,18 @@ _mongoc_cse_client_pool_enable_auto_encryption (
14081424
topology->keyvault_client_pool = opts->keyvault_client_pool;
14091425
}
14101426

1411-
ret = true;
1427+
setup_okay = true;
1428+
BSON_ASSERT (prev_cse_state == MONGOC_CSE_DISABLED);
14121429
fail:
1430+
if (prev_cse_state == MONGOC_CSE_DISABLED) {
1431+
/* We need to set the new CSE state. */
1432+
mongoc_topology_cse_state_t new_state =
1433+
setup_okay ? MONGOC_CSE_ENABLED : MONGOC_CSE_DISABLED;
1434+
bson_atomic_int_exchange (
1435+
(int *) &topology->cse_state, new_state, bson_memory_order_release);
1436+
}
14131437
mongoc_uri_destroy (mongocryptd_uri);
1414-
bson_mutex_unlock (&topology->mutex);
1415-
RETURN (ret);
1438+
RETURN (setup_okay);
14161439
}
14171440

14181441
struct _mongoc_client_encryption_t {
@@ -1659,12 +1682,15 @@ mongoc_client_encryption_decrypt (mongoc_client_encryption_t *client_encryption,
16591682
bool
16601683
_mongoc_cse_is_enabled (mongoc_client_t *client)
16611684
{
1662-
bool ret = false;
1663-
1664-
bson_mutex_lock (&client->topology->mutex);
1665-
ret = client->topology->cse_enabled;
1666-
bson_mutex_unlock (&client->topology->mutex);
1667-
return ret;
1685+
while (1) {
1686+
mongoc_topology_cse_state_t state = bson_atomic_int_fetch (
1687+
(int *) &client->topology->cse_state, bson_memory_order_relaxed);
1688+
if (state != MONGOC_CSE_STARTING) {
1689+
return state == MONGOC_CSE_ENABLED;
1690+
}
1691+
/* CSE is starting up. Wait until that succeeds or fails. */
1692+
bson_thrd_yield ();
1693+
}
16681694
}
16691695

16701696
#endif /* MONGOC_ENABLE_CLIENT_SIDE_ENCRYPTION */

0 commit comments

Comments
 (0)