Skip to content

CDRIVER-4425 Provide access to raw result if server error occurs #1144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 29 commits into from
Nov 16, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
95ba2c7
CDRIVER-4425 new test files and parse option
galon1 Nov 7, 2022
077af67
CDRIVER-4425 remove changes to bson_error_t
galon1 Nov 9, 2022
8208cad
CDRIVER-4425 add errorResponse handling in test runner
galon1 Nov 9, 2022
9781c97
CDRIVER-4425 removing unintended changes
galon1 Nov 9, 2022
ff8bf0e
CDRIVER-4425 insertOne test passes
galon1 Nov 10, 2022
36a695a
CDRIVER-4425 add unique field and optional name to createIndex
galon1 Nov 10, 2022
faaf72b
CDRIVER-4425 remove unintended changes and add comment
galon1 Nov 10, 2022
2a4e901
CDRIVER-4425 add check for result->error is server
galon1 Nov 10, 2022
76514d8
CRIVER-4425 fix comment
galon1 Nov 10, 2022
693bcbb
CDRIVER-4425 add linebreak back
galon1 Nov 10, 2022
19068c1
CDRIVER-4425 fix comment
galon1 Nov 10, 2022
cfe57cb
CDRIVER-4425 wrap write operation return in serverResponse
galon1 Nov 10, 2022
13601c5
CDRIVER-4425 destroy before copying to avoid leak
galon1 Nov 10, 2022
44f0450
CDRIVER-4425 add destroy to avoid memory leak
galon1 Nov 10, 2022
c52b2b1
CDRIVER-4425 do not include unique in createIndex if not set
galon1 Nov 11, 2022
3435887
CDRIVER-4425 make parameter constant
galon1 Nov 11, 2022
1de331c
CDRIVER-4425 store error replies as an array
galon1 Nov 11, 2022
78629b1
CDRIVER-4425 fix comment and delete unintended file
galon1 Nov 11, 2022
8d4b644
Merge branch 'master' of github.com:galon1/mongo-c-driver into CDRIVE…
galon1 Nov 11, 2022
faf48b1
CDRIVER-4425 change check to errorReplies, check null doc in test, an…
galon1 Nov 14, 2022
abc4c7e
CDRIVER-4425 change how createIndex command is built
galon1 Nov 14, 2022
f9ece40
CDRIVER-4425 test with multiple errorReplies
galon1 Nov 14, 2022
69e6773
CDRIVER-4425 change assert_error_count name for clarity
galon1 Nov 14, 2022
8314294
CDRIVER-4425 move destroy statements to avoid memory leak
galon1 Nov 14, 2022
b74b2da
CDRIVER-4425 skip test if no failpoint, remove leak
galon1 Nov 14, 2022
ba4068f
CDRIVER-4425 remove unintended comment
galon1 Nov 14, 2022
76fa9e5
CDRIVER-4425 typecheck result from write responses
galon1 Nov 15, 2022
73f10aa
CDRIVER-4425 fix comment
galon1 Nov 15, 2022
54abd64
CDRIVER-4425 move child inside if statement
galon1 Nov 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions src/libmongoc/src/mongoc/mongoc-error-private.h
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,9 @@ _mongoc_error_is_state_change (bson_error_t *error);
bool
_mongoc_error_is_network (const bson_error_t *error);

bool
_mongoc_error_is_server (const bson_error_t *error);

bool
_mongoc_error_is_auth (const bson_error_t *error);

Expand Down
4 changes: 2 additions & 2 deletions src/libmongoc/src/mongoc/mongoc-error.c
Original file line number Diff line number Diff line change
Expand Up @@ -59,8 +59,8 @@ mongoc_error_has_label (const bson_t *reply, const char *label)
return false;
}

static bool
_mongoc_error_is_server (bson_error_t *error)
bool
_mongoc_error_is_server (const bson_error_t *error)
{
if (!error) {
return false;
Expand Down
3 changes: 3 additions & 0 deletions src/libmongoc/src/mongoc/mongoc-write-command-private.h
Original file line number Diff line number Diff line change
Expand Up @@ -88,6 +88,9 @@ typedef struct {
* primary, this contains the server id of the newly selected primary. Only
* applies to OP_MSG. Is left at 0 if no retry occurs. */
uint32_t retry_server_id;
/* store the raw server response if an error occured */
uint32_t n_errorReplies;
bson_t rawErrorReplies;
} mongoc_write_result_t;


Expand Down
25 changes: 25 additions & 0 deletions src/libmongoc/src/mongoc/mongoc-write-command.c
Original file line number Diff line number Diff line change
Expand Up @@ -1147,6 +1147,7 @@ _mongoc_write_result_init (mongoc_write_result_t *result) /* IN */
bson_init (&result->writeConcernErrors);
bson_init (&result->writeErrors);
bson_init (&result->errorLabels);
bson_init (&result->rawErrorReplies);

EXIT;
}
Expand All @@ -1163,6 +1164,7 @@ _mongoc_write_result_destroy (mongoc_write_result_t *result)
bson_destroy (&result->writeConcernErrors);
bson_destroy (&result->writeErrors);
bson_destroy (&result->errorLabels);
bson_destroy (&result->rawErrorReplies);

EXIT;
}
Expand Down Expand Up @@ -1355,6 +1357,22 @@ _mongoc_write_result_merge (mongoc_write_result_t *result, /* IN */
result->n_writeConcernErrors++;
}

/* If a server error ocurred, then append the raw response to the
* error_replies array. */
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Offhand, would the first { ok: 0 } command response (post-retry) completely interrupt a bulk write operation regardless of the ordered option (note: ordered: false is essentially continue-on-error).

Our specs are ambiguous about this (see: DRIVERS-2090, DRIVERS-2159).

I noted in the unified test runner code below, the assertion is only done on errorReplies.0, which assumes that three will only ever be on error reply. That's fine, as I intentionally wrote the tests with the above ambiguity in mind (see: mongodb/specifications@33a03f2.

But here, I'm more curious about whether multiple errorReplies are even possible in libmongoc. If not, are we just using an errorReplies array on the result for flexibility?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is possible for libmongoc to return multiple errorReplies. Here is a test displaying this behavior. Since ordered: false continues on error when the failPoint is hit twice, two errorReplies are returned by libmongoc, as well as other errors that occurred if they did (in the test a writeError also was raised).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for indulging my question. Looks like an oversight in the test runner syntax I introduced in DRIVERS-2385. I've made a note about this in DRIVERS-1547 as that's the most likely ticket we'd use to add spec test for multiple command failures in a bulk write.

if (!_mongoc_cmd_check_ok (
reply, MONGOC_ERROR_API_VERSION_2, NULL /* error */)) {
char str[16];
const char *key;

bson_uint32_to_string (result->n_errorReplies, &key, str, sizeof str);

if (!bson_append_document (&result->rawErrorReplies, key, -1, reply)) {
MONGOC_ERROR ("Error adding \"%s\" to errorReplies.\n", key);
}

result->n_errorReplies++;
}

/* inefficient if there are ever large numbers: for each label in each err,
* we linear-search result->errorLabels to see if it's included yet */
_mongoc_bson_array_copy_labels_to (reply, &result->errorLabels);
Expand Down Expand Up @@ -1465,6 +1483,7 @@ _mongoc_write_result_complete (
/* produce either old fields like nModified from the deprecated Bulk API Spec
* or new fields like modifiedCount from the CRUD Spec, which we partly obey
*/

if (bson && mongoc_write_concern_is_acknowledged (wc)) {
n_args = 0;
va_start (args, error);
Expand Down Expand Up @@ -1529,6 +1548,12 @@ _mongoc_write_result_complete (
}
}

/* If there is a raw error response then we know a server error has occurred.
* We should add the raw result to the reply. */
if (bson && !bson_empty (&result->rawErrorReplies)) {
BSON_APPEND_ARRAY (bson, "errorReplies", &result->rawErrorReplies);
}

/* set bson_error_t from first write error or write concern error */
_set_error_from_response (
&result->writeErrors, domain, "write", &result->error);
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
{
"description": "modifyCollection-errorResponse",
"schemaVersion": "1.12",
"createEntities": [
{
"client": {
"id": "client0",
"observeEvents": [
"commandStartedEvent"
]
}
},
{
"database": {
"id": "database0",
"client": "client0",
"databaseName": "collMod-tests"
}
},
{
"collection": {
"id": "collection0",
"database": "database0",
"collectionName": "test"
}
}
],
"initialData": [
{
"collectionName": "test",
"databaseName": "collMod-tests",
"documents": [
{
"_id": 1,
"x": 1
},
{
"_id": 2,
"x": 1
}
]
}
],
"tests": [
{
"description": "modifyCollection prepareUnique violations are accessible",
"runOnRequirements": [
{
"minServerVersion": "5.2"
}
],
"operations": [
{
"name": "createIndex",
"object": "collection0",
"arguments": {
"keys": {
"x": 1
}
}
},
{
"name": "modifyCollection",
"object": "database0",
"arguments": {
"collection": "test",
"index": {
"keyPattern": {
"x": 1
},
"prepareUnique": true
}
}
},
{
"name": "insertOne",
"object": "collection0",
"arguments": {
"document": {
"_id": 3,
"x": 1
}
},
"expectError": {
"errorCode": 11000
}
},
{
"name": "modifyCollection",
"object": "database0",
"arguments": {
"collection": "test",
"index": {
"keyPattern": {
"x": 1
},
"unique": true
}
},
"expectError": {
"isClientError": false,
"errorCode": 359,
"errorResponse": {
"violations": [
{
"ids": [
1,
2
]
}
]
}
}
}
]
}
]
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
{
"description": "aggregate-merge-errorResponse",
"schemaVersion": "1.12",
"createEntities": [
{
"client": {
"id": "client0"
}
},
{
"database": {
"id": "database0",
"client": "client0",
"databaseName": "crud-tests"
}
},
{
"collection": {
"id": "collection0",
"database": "database0",
"collectionName": "test"
}
}
],
"initialData": [
{
"collectionName": "test",
"databaseName": "crud-tests",
"documents": [
{
"_id": 1,
"x": 1
},
{
"_id": 2,
"x": 1
}
]
}
],
"tests": [
{
"description": "aggregate $merge DuplicateKey error is accessible",
"runOnRequirements": [
{
"minServerVersion": "5.1",
"topologies": [
"single",
"replicaset"
]
}
],
"operations": [
{
"name": "aggregate",
"object": "database0",
"arguments": {
"pipeline": [
{
"$documents": [
{
"_id": 2,
"x": 1
}
]
},
{
"$merge": {
"into": "test",
"whenMatched": "fail"
}
}
]
},
"expectError": {
"errorCode": 11000,
"errorResponse": {
"keyPattern": {
"_id": 1
},
"keyValue": {
"_id": 2
}
}
}
}
]
}
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
{
"description": "bulkWrite-errorResponse",
"schemaVersion": "1.12",
"createEntities": [
{
"client": {
"id": "client0",
"useMultipleMongoses": false
}
},
{
"database": {
"id": "database0",
"client": "client0",
"databaseName": "crud-tests"
}
},
{
"collection": {
"id": "collection0",
"database": "database0",
"collectionName": "test"
}
}
],
"tests": [
{
"description": "bulkWrite operations support errorResponse assertions",
"runOnRequirements": [
{
"minServerVersion": "4.0.0",
"topologies": [
"single",
"replicaset"
]
},
{
"minServerVersion": "4.2.0",
"topologies": [
"sharded"
]
}
],
"operations": [
{
"name": "failPoint",
"object": "testRunner",
"arguments": {
"client": "client0",
"failPoint": {
"configureFailPoint": "failCommand",
"mode": {
"times": 1
},
"data": {
"failCommands": [
"insert"
],
"errorCode": 8
}
}
}
},
{
"name": "bulkWrite",
"object": "collection0",
"arguments": {
"requests": [
{
"insertOne": {
"document": {
"_id": 1
}
}
}
]
},
"expectError": {
"errorCode": 8,
"errorResponse": {
"code": 8
}
}
}
]
}
]
}

Loading