Skip to content

DOCSP-41887 Async Migration Guide #103

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conversation

jordan-smith721
Copy link
Collaborator

@jordan-smith721 jordan-smith721 commented Aug 30, 2024

Pull Request Info

PR Reviewing Guidelines

JIRA - https://jira.mongodb.org/browse/DOCSP-41887
Staging -
https://deploy-preview-103--docs-pymongo.netlify.app/motor-async-migration/
https://deploy-preview-103--docs-pymongo.netlify.app/pymongo-to-async-guide

Self-Review Checklist

  • Is this free of any warnings or errors in the RST?
  • Did you run a spell-check?
  • Did you run a grammar-check?
  • Are all the links working?
  • Are the facets and meta keywords accurate?

Copy link

netlify bot commented Aug 30, 2024

Deploy Preview for docs-pymongo ready!

Name Link
🔨 Latest commit bf38a06
🔍 Latest deploy log https://app.netlify.com/sites/docs-pymongo/deploys/670e80f656934f000844f0aa
😎 Deploy Preview https://deploy-preview-103--docs-pymongo.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@jordan-smith721 jordan-smith721 marked this pull request as ready for review September 6, 2024 17:25
@jordan-smith721 jordan-smith721 changed the title draft start DOCSP-41887 Async Migration Guide Sep 6, 2024
Copy link
Collaborator

@rustagir rustagir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few things!

source/index.txt Outdated
Migrate to the {+driver-async+} Driver
------------------------------------

Learn how to migrate from {+driver-short+} or Motor to the {+driver-async+} driver in the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S: I think another sentence here could be useful before linking to the guide. Maybe you can mention the time period that this applies to?

Eg.
In Aug 2024, MongoDB released the PyMongo Async driver to unify PyMongo and Motor, the asynchronous MongoDB driver for Python.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flagging this again, but also ok if you wanted to leave as is!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, misread this the first time and applied it to the wrong area. Fixed!

Copy link
Collaborator

@rustagir rustagir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

approving as comments arent blocking but happy to re-review if needed!

source/index.txt Outdated
Migrate to the {+driver-async+} Driver
------------------------------------

Learn how to migrate from {+driver-short+} or Motor to the {+driver-async+} driver in the
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Flagging this again, but also ok if you wanted to leave as is!

Comment on lines 60 to 65
* - ``AsyncMongoClient()``
- .. code-block:: python

from pymongo import AsyncMongoClient

client = AsyncMongoClient(...)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S: perhaps this should have its own subsection as its a constructor rather than a method?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought about that at first, but it seemed weird to be in its own table and it's related to the client so I figured it made sense here. Can change if anyone has strong feelings the other way though

Copy link

@NoahStapp NoahStapp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First pass, I'll take another look tomorrow morning for a deeper dive on the technical details.

source/index.txt Outdated

In September 2024, MongoDB released the {+driver-async+} driver to unify {+driver-short+}
and `Motor <https://www.mongodb.com/docs/drivers/motor/>`__, the asynchronous
MongoDB driver for Python. Learn how to migrate from {+driver-short+} or Motor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should be clear that the new Async API is a replacement for Motor specifically and that any existing users of PyMongo will only have to migrate if they want to switch to async.


The {+driver-async+} driver is a unification of {+driver-short+} and the `Motor
library <https://www.mongodb.com/docs/drivers/motor/>`__. In this guide, you can
identify the changes you must make to migrate an application from

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, the word "migrate" seems a little confusing for existing PyMongo users. Is there a different wording we can use to make sure users understand that they only need to make changes if they want to use the new async API?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reworded this after splitting pages

- Add the ``await`` keyword to all asynchronous method calls.
- If an asynchronous method is called within a function, mark the function as ``async``.

The following sections describe how to implement the asynchronous API.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd also link to the official Python async docs here for some additional context.


from pymongo import AsyncMongoClient

AsyncMongoClient(...)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To use an AsyncMongoClient as a context manager users will have to do async with AsyncMongoClient(...) instead of with MongoClient(...).

Migrate From Motor
------------------

The {+driver-async+} driver behaves nearly identically to the Motor library. In
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a little misleading. Async PyMongo is (or should be) vastly superior to Motor in behavior both in terms of latency and throughput because Async Pymongo directly uses asyncio whereas Motor delegates all work to a thread pool.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the wording here, let me know if that works better!


await collection.find_one_and_update(...)

Migrate From Motor
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to see the motor -> async pymongo migration as its own dedicated page. The pymongo sync -> async differences clutters the page for motor users.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Split the pages into two

Copy link

@NoahStapp NoahStapp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One clarifying suggestion, otherwise looks good!


- Replace all uses of ``MongoClient`` with ``AsyncMongoClient``.
- Add the ``await`` keyword to all asynchronous method calls.
- If an asynchronous method is called within a function, mark the function as ``async``.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's unclear that this last point is for functions that the user defines.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call. Updated

@jordan-smith721
Copy link
Collaborator Author

@ShaneHarvey any other suggested changes on your end before I merge?

source/index.txt Outdated
Migrate from Motor to {+driver-async+}
-------------------------------------

In September 2024, MongoDB released the {+driver-async+} driver as a replacement
Copy link
Member

@ShaneHarvey ShaneHarvey Sep 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The Async PyMongo is released yes, but it's only in beta (ie experimental) and not ready for production use. We should make this loud and clear one every one of these pages.


- ``GridOut.open()`` returns ``None``.
- ``AsyncMongoClient.__init__()`` does not accept an ``io_loop`` parameter.
- ``GridIn.set()`` does not accept a filename. Instead pass a file name by using the
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@noah is this true? Does GridIn here refer to a motor class? The pymongo class is AsyncGridIn.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AsyncGridIn.set() does accept filenames.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were from the original draft migration notes linked in the ticket. Should I delete the GridIn bullets?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


The following Motor method signatures behave differently in the {+driver-async+} driver:

- ``GridOut.open()`` returns ``None``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does GridOut here refer to a motor class? The pymongo class is AsyncGridOut.

- ``AsyncMongoClient.__init__()`` does not accept an ``io_loop`` parameter.
- ``GridIn.set()`` does not accept a filename. Instead pass a file name by using the
``GridIn.filename`` attribute.
- ``Cursor.each()`` does not exist in the {+driver-async+} driver.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor -> AsyncCursor.

- ``GridIn.set()`` does not accept a filename. Instead pass a file name by using the
``GridIn.filename`` attribute.
- ``Cursor.each()`` does not exist in the {+driver-async+} driver.
- ``stream_to_handler()`` does not exist in the {+driver-async+} driver.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MotorGridOut.stream_to_handler()

``GridIn.filename`` attribute.
- ``Cursor.each()`` does not exist in the {+driver-async+} driver.
- ``stream_to_handler()`` does not exist in the {+driver-async+} driver.
- ``to_list(0)`` is not valid in the {+driver-async+} driver. Use
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AsyncCursor.to_list(0)


- Replace all uses of ``MongoClient`` with ``AsyncMongoClient``.
- Add the ``await`` keyword to all asynchronous method calls.
- If you call an asynchronous method inside of a function, mark the function as ``async``.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add:

- A ``MongoClient`` is thread safe and can be used be many threads. A ``AsyncMongoClient`` is not thread safe and should only be used by a single event loop. 

@jordan-smith721 jordan-smith721 merged commit acff81f into mongodb:master Oct 22, 2024
4 of 5 checks passed
@jordan-smith721 jordan-smith721 deleted the DOCSP-41887-pymongoasync-migration-guide branch October 22, 2024 14:07
jordan-smith721 added a commit that referenced this pull request Oct 22, 2024
jordan-smith721 added a commit that referenced this pull request Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants