Skip to content

DOCSP-47835 Clarify batchsize behavior #215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 17 commits into from
Mar 11, 2025
Merged

DOCSP-47835 Clarify batchsize behavior #215

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
4d29070
change batchsize wording
shuangela Mar 5, 2025
bf79b9e
remove brackets
shuangela Mar 6, 2025
b02a39a
remove brackets
shuangela Mar 6, 2025
248115f
fix typo
shuangela Mar 6, 2025
6f46523
change formatting
shuangela Mar 6, 2025
a7bd71c
clarify wording based on rr feedback
shuangela Mar 6, 2025
269819f
change based on feedback
shuangela Mar 7, 2025
76d9bd5
clarify wording from jm feedback
shuangela Mar 7, 2025
69d3ed4
fix format
shuangela Mar 7, 2025
c18ef44
fix typo
shuangela Mar 7, 2025
e9aab32
rr feedback
shuangela Mar 10, 2025
6183d86
rr and jm feedback
shuangela Mar 10, 2025
89b304c
typo
shuangela Mar 10, 2025
635e787
style guide
shuangela Mar 10, 2025
debcfe1
fix typos
shuangela Mar 10, 2025
380a16e
fix rr
shuangela Mar 10, 2025
892f7ea
jm review
shuangela Mar 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions source/includes/extracts-watch-option.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
ref: watch-option-batchSize
content: |
Specifies the batch size for the cursor, which will apply to both the initial
``aggregate`` command and any subsequent ``getMore`` commands. This determines
the maximum number of change events to return in each response from the
server.
The maximum number of documents within each batch returned in a query result. By default,
the ``find()`` method has an initial batch size of ``101`` documents
and a maximum of 16 megabytes for each subsequent batch. This
option can enforce a smaller limit than 16 megabytes, but not a larger
one. If you set ``batchSize`` to a limit that results in batches larger than
16 MB, this option has no effect.
Copy link
Member

@jmikola jmikola Mar 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change streams have nothing to do with the find() method. I think we should preserve the original wording that referred to aggregate and "change events", since this is a snippet for watch() methods.

These use the aggregate command and $changeStream stage. A batch size for aggregate is expressed via the cursor.batchSize option on the outgoing command. For getMore commands (common to both aggregate and find), batchSize is a top-level option.

In the interest of correctness and consistency (with other docs), we should refer to "mebibytes" or "MiB" instead of "megabytes" and "MB".

Copy link
Member

@jmikola jmikola Mar 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding on to this, I think it'd be more technically correct to refer to "aggregate command" instead of "watch() method". That helps communicate to users that this driver method is actually running an aggregate command on the server, and I think it's helpful since they can then cross-reference with the server's own API docs for commands.

I understand this might not be desirable in prose-style docs (e.g. read/retrieve.txt), but this snippet is included in our API reference.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, left a question above about MB vs MiB but saw you already answered it here! Feel free to ignore.

Copy link
Collaborator Author

@shuangela shuangela Mar 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jmikola For clarification, in this comment "Adding on to this, I think it'd be more technically correct to refer to "aggregate command" instead of "watch() method". That helps communicate to users that this driver method is actually running an aggregate command on the server, and I think it's helpful since they can then cross-reference with the server's own API docs for commands," when you say the "watch() method" did you mean the "find() method"? I don't think this doc refers to the watch() method. Just wanted to clarify to be sure I'm understanding correctly!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file is extracts-watch-option.yaml and included in the docs for watch() methods on Client, Database, and Collection objects.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, I think I misunderstood the comment and understand what you're saying now. Thanks for the help!


Irrespective of the ``batchSize`` option, the initial ``aggregate`` command
response for a change stream generally does not include any documents
Expand Down
7 changes: 6 additions & 1 deletion source/read/retrieve.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,12 @@ you can set in the array:
- Description

* - ``batchSize``
- | The number of documents to return per batch. The default value is ``101``.
- | The maximum number of documents within each batch returned in a query result. By default,
the ``find()`` method has an initial batch size of ``101`` documents
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this section is talking about both find() and findOne(), I think this can just refer to "the find command" (used by both helpers).

Therefore, this snippet will just duplicate the first paragraph from the batchSize docs in MongoDBCollection-find.txt.

and a maximum of 16 megabytes for each subsequent batch. This
option can enforce a smaller limit than 16 megabytes, but not a larger
one. If you set ``batchSize`` to a limit that results in batches larger than
16 MB, this option has no effect.
| **Type**: ``integer``

* - ``collation``
Expand Down
9 changes: 6 additions & 3 deletions source/reference/method/MongoDBCollection-find.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,9 +52,12 @@ Parameters

* - batchSize
- integer
- The number of documents to return in the first batch. Defaults to
``101``. A batchSize of ``0`` means that the cursor will be
established, but no documents will be returned in the first batch.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The special behavior of batchSize: 0 is notable, and also discussed in the find command docs. Particularly with aggregate operations, there is a legitimate use case for wanting to start the aggregation on the server without waiting for the first result to become available. Allowing a cursor to be created immediately and iterated later (via batchSize: 0 satisfies that).

- The maximum number of documents within each batch returned in a query result. By default,
the ``find()`` method has an initial batch size of ``101`` documents
and a maximum of 16 megabytes for each subsequent batch. This
option can enforce a smaller limit than 16 megabytes, but not a larger
one. If you set ``batchSize`` to a limit that results in batches larger than
16 MB, this option has no effect.

Unlike the previous wire protocol version, a batchSize of ``1`` for the
:dbcommand:`find` command does not close the cursor.
Expand Down
10 changes: 6 additions & 4 deletions source/reference/method/MongoDBCollection-listSearchIndexes.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,12 @@ Parameters

* - batchSize
- integer
- Specifies the batch size for the cursor, which will apply to both the
initial ``aggregate`` command and any subsequent ``getMore`` commands.
This determines the maximum number of documents to return in each
response from the server.
- The maximum number of documents within each batch returned in a query result. By default,
the ``find()`` method has an initial batch size of ``101`` documents
and a maximum of 16 megabytes for each subsequent batch. This
option can enforce a smaller limit than 16 megabytes, but not a larger
one. If you set ``batchSize`` to a limit that results in batches larger than
16 MB, this option has no effect.

A batchSize of ``0`` is special in that and will only apply to the
initial ``aggregate`` command; subsequent ``getMore`` commands will use
Expand Down
12 changes: 7 additions & 5 deletions source/reference/method/MongoDBDatabase-aggregate.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,13 @@ Parameters

* - batchSize
- integer
- Specifies the batch size for the cursor, which will apply to both the
initial ``aggregate`` command and any subsequent ``getMore`` commands.
This determines the maximum number of documents to return in each
response from the server.
Copy link
Member

@jmikola jmikola Mar 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can discuss batchSize: 0 here as we do for the find() option, since there is a legitimate use case for doing so. I'm OK with keeping this in the API reference and out of the prose docs (e.g. read/retrieve.txt), since it's more technical.

There is also a special directive in the CRUD spec for drivers to intentionally omit batchSize for pipelines that include $out or $merge, since it might prevent the pipeline from executing (and writing output). I don't think we need to mention that, though, as it is enforced internally (see: Aggregate.php).


- The maximum number of documents within each batch returned in a query result. By default,
the ``find()`` method has an initial batch size of ``101`` documents
and a maximum of 16 megabytes for each subsequent batch. This
option can enforce a smaller limit than 16 megabytes, but not a larger
one. If you set ``batchSize`` to a limit that results in batches larger than
16 MB, this option has no effect.

A batchSize of ``0`` is special in that and will only apply to the
initial ``aggregate`` command; subsequent ``getMore`` commands will use
the server's default batch size. This may be useful for quickly
Expand Down
Loading