(DOCS-13431): Multi-document transactions can implicitly create collections

Author: jeff-allen-mongo

source/core/transactions-in-applications.txt

@@ -117,7 +117,7 @@ The new callback API incorporates logic:
              final MongoClient client = MongoClients.create(uri);
 
              /*
-                 Prereq: Create collections. CRUD operations in transactions must be on existing collections.
+                 Create collections:
               */
 
              client.getDatabase("mydb1").getCollection("foo")
@@ -453,7 +453,7 @@ transactions:
 .. code-block:: javascript
    :copyable: false
 
-   // Prereq: Create collections. CRUD operations in transactions must be on existing collections.
+   // Create collections:
    db.getSiblingDB("mydb1").foo.insert( {abc: 0}, { writeConcern: { w: "majority", wtimeout: 2000 } } );
    db.getSiblingDB("mydb2").bar.insert( {xyz: 0}, { writeConcern: { w: "majority", wtimeout: 2000 } } );
 
@@ -480,4 +480,3 @@ transactions:
    session.commitTransaction();
 
    session.endSession();
-

source/core/transactions.txt

@@ -98,7 +98,7 @@ of the transactions API:
          final MongoClient client = MongoClients.create(uri);
 
          /*
-             Prereq: Create collections. CRUD operations in transactions must be on existing collections.
+             Create collections:
           */
 
          client.getDatabase("mydb1").getCollection("foo")
@@ -342,10 +342,7 @@ For transactions:
 
 .. include:: /includes/extracts/transactions-operations-crud.rst
 
-Operations that affect the database catalog, such as creating or
-dropping a collection or an index, are not allowed in
-transactions. For example, a transaction cannot include an insert
-operation that would result in the creation of a new collection. See
+For a list of operations not supported in transactions, see
 :ref:`transactions-ops-restricted`.
 
 .. include:: /includes/extracts/transactions-operations-catalog-tip.rst
@@ -354,6 +351,23 @@ operation that would result in the creation of a new collection. See
 
    - :doc:`Transactions and Operations Reference </core/transactions-operations>`
 
+.. _transactions-create-collections-indexes:
+
+Create Collections and Indexes In a Transaction
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 4.4
+
+When using :ref:`feature compatibility version (fcv) <view-fcv>`
+``"4.4"`` or greater, you can create a collection and index
+within a transaction. Specifically, you can:
+
+.. include:: /includes/extracts/transactions-4.4-allowed-operations.rst
+
+When using fcv ``"4.2"`` or less, no operations that affect the
+database catalog, such as creating or dropping a collection or an index,
+are allowed in transactions.
+
 .. _transactions-ops-count:
 
 Count Operation

source/includes/extracts-4.4-changes.yaml

@@ -311,4 +311,63 @@ content: |
    By default MongoDB does not set a minimum oplog retention period and
    automatically truncates the oplog starting with the oldest entries to
    maintain the configured maximum oplog size.
+---
+ref: 4.4-changes-transactions-upsert
+content: |
+
+   For :ref:`feature compatibility version (fcv) <view-fcv>` ``"4.4"``
+   and greater, if the operation is specified on a non-existing
+   collection and results in an upsert, the collection is implicitly
+   created.
+
+   .. note::
+
+      .. include:: /includes/extracts/transactions-cross-shard-collection-restriction.rst
+
+   For fcv ``"4.2"`` or less, if the operation results in an upsert,
+   the collection must already exist.
+---
+ref: 4.4-changes-transactions-insert
+content: |
+
+   For :ref:`feature compatibility version (fcv) <view-fcv>` ``"4.4"``
+   and greater, if you specify an insert on a non-existing collection in
+   a transaction, the collection is implicitly created.
+
+   .. note::
+
+      .. include:: /includes/extracts/transactions-cross-shard-collection-restriction.rst
+
+   For fcv ``"4.2"`` or less, insert operations that would result in
+   the creation of a new collection are not allowed in a transaction.
+---
+ref: 4.4-changes-transactions-bulkWrite
+content: |
+
+   For :ref:`feature compatibility version (fcv) <view-fcv>` ``"4.4"``
+   and greater, if an insert operation or update operation with
+   ``upsert: true`` is run in a transaction against a non-existing
+   collection, the collection is implicitly created.
+
+   .. note::
+
+      .. include:: /includes/extracts/transactions-cross-shard-collection-restriction.rst
+
+   For fcv ``"4.2"`` or less, the collection must already exist for
+   insert and ``upsert: true`` operations.
+---
+ref: 4.4-changes-transactions-save
+content: |
+
+   For :ref:`feature compatibility version (fcv) <view-fcv>` ``"4.4"``
+   and greater, if you save a document to a non-existing collection in a
+   transaction, the collection is implicitly created.
+
+   .. note::
+
+      .. include:: /includes/extracts/transactions-cross-shard-collection-restriction.rst
+
+   For fcv ``"4.2"`` or less, :method:`~db.collection.save()`
+   operations that would result in the creation of a new collection are
+   not allowed in a transaction.
 ...

source/includes/extracts-transactions.yaml

@@ -80,21 +80,56 @@ content: |
 ---
 ref: transactions-operations-restrictions
 content: |
+
+   .. versionchanged:: 4.4
  
    The following operations are not allowed in transactions:
 
-   - Operations that affect the database catalog, such as creating or
-     dropping a collection or an index. For example, a
-     transaction cannot include an insert operation that would result
-     in the creation of a new collection.
-
-     The :dbcommand:`listCollections` and :dbcommand:`listIndexes`
-     commands and their helper methods are also excluded.
+   - Operations that affect the database catalog,
+     such as creating or dropping a collection or an index, unless
+     using :ref:`feature compatibility version (fcv) <view-fcv>`
+     ``"4.4"`` or greater. With fcv ``"4.4"`` or greater you can create
+     collections and indexes in transactions. For details, see
+     :ref:`transactions-create-collections-indexes`.
  
-   - Non-CRUD and non-informational operations, such as
+   - Creating new collections in cross-shard write transactions. For
+     example, if you write to an existing collection in one shard and
+     implicitly create a collection in a different shard, MongoDB cannot
+     perform both operations in the same transaction.
+
+   - The :dbcommand:`listCollections` and :dbcommand:`listIndexes`
+     commands and their helper methods.
+
+   - Other non-CRUD and non-informational operations, such as
      :dbcommand:`createUser`, :dbcommand:`getParameter`,
      :dbcommand:`count`, etc. and their helpers.
 
+---
+ref: transactions-4.4-allowed-operations
+content: |
+
+   - Create a collection in a transaction.
+
+     - Explicitly create a collection in a transaction using
+       :method:`db.createCollection()`.
+
+     - Implicitly create a collection in a transaction by running an
+       insert operation or update operation with ``upsert: true``
+       against a non-existing collection.
+
+     .. note::
+
+        .. include:: /includes/extracts/transactions-cross-shard-collection-restriction.rst
+
+   - Create an index using :dbcommand:`createIndexes` on:
+
+     - A non-existing collection. In this case, the collection is
+       created implicitly.
+
+     - An empty collection which was created earlier in the same
+       multi-document transaction.
+
+   
 ---
 ref: transactions-operations-restrictions-info
 content: |
@@ -109,8 +144,20 @@ ref: transactions-operations-crud
 content: |
 
    - You can specify read/write (CRUD) operations on **existing**
-     collections. The collections can be in different databases. For a
-     list of CRUD operations, see :ref:`transactions-operations-crud`.
+     collections. For a list of CRUD operations, see
+     :ref:`transactions-operations-crud`.
+  
+   - When using :ref:`feature compatibility version (fcv) <view-fcv>`
+     ``"4.4"`` or greater, you can create collections and indexes
+     in transactions. For details, see
+     :ref:`transactions-create-collections-indexes`
+   
+   - The collections used in a transaction can be in different
+     databases.
+
+     .. note::
+      
+        .. include:: /includes/extracts/transactions-cross-shard-collection-restriction.rst
 
    - You cannot write to :doc:`capped </core/capped-collections>`
      collections. (Starting in MongoDB 4.2)
@@ -127,6 +174,14 @@ content: |
    - .. include:: /includes/extracts/transactions-killop-change.rst
 
 ---
+ref: transactions-cross-shard-collection-restriction
+content: |
+
+   You cannot create new collections in cross-shard write transactions.
+   For example, if you write to an existing collection in one shard and
+   implicitly create a collection in a different shard, MongoDB cannot
+   perform both operations in the same transaction.
+---
 ref: transactions-operations-catalog-tip
 content: |
 
@@ -502,4 +557,3 @@ content: |
    sharded cluster transaction includes some parts that use ``{w:
    "majority", j: true}`` write concern.
 ...
-

source/includes/table-transactions-operations.rst

@@ -69,22 +69,36 @@
 
      - :dbcommand:`insert`
 
-     - Only when run against an existing collection.
+     - For :ref:`feature compatibility version (fcv) <view-fcv>`
+       ``"4.4"`` or greater, when run against a non-existing
+       collection, the collection is implicitly created.
+       
+       For fcv ``"4.2"`` or less, can only be run against an existing
+       collection.
 
    * - :method:`db.collection.save()`
      - 
-     - If an insert, only when run against an existing collection.
+     - For :ref:`feature compatibility version (fcv) <view-fcv>`
+       ``"4.4"`` or greater, if an insert against a non-existing
+       collection, the collection is implicitly created.
+       
+       With fcv ``"4.2"`` or less, if an insert, can only be run against an
+       existing collection.
 
    * - | :method:`db.collection.updateOne()`
        | :method:`db.collection.updateMany()`
        | :method:`db.collection.replaceOne()`
        | :method:`db.collection.update()`
 
      - :dbcommand:`update`
-     - For ``upsert``, only when run against an existing collection.
+     - For :ref:`feature compatibility version (fcv) <view-fcv>`
+       ``"4.4"`` or greater, if run with ``upsert: true`` against a
+       non-existing collection, the collection is implicitly created.
+
+       For fcv ``"4.2"`` or less, if ``upsert: true``, the operation must be
+       run against an existing collection.
 
    * - | :method:`db.collection.bulkWrite()`
        | Various :doc:`/reference/method/js-bulk`
      - 
-     - | For insert operations, only when run against an existing collection.
-       | For ``upsert``, only when run against an existing collection.
+     - .. include:: /includes/extracts/4.4-changes-transactions-bulkWrite.rst

source/reference/command/insert.txt

@@ -125,9 +125,7 @@ Transactions
 
 .. include:: /includes/extracts/transactions-supported-operation.rst
 
-The collection must already exist. An insert operation that
-would result in the creation of a new collection are not allowed in a
-transaction.
+.. include:: /includes/extracts/4.4-changes-transactions-insert.rst
 
 .. include:: /includes/extracts/transactions-operations-write-concern.rst
 

source/reference/command/update.txt

@@ -564,7 +564,7 @@ Transactions
 
 .. include:: /includes/extracts/transactions-supported-operation.rst
 
-If the operation results in an upsert, the collection must already exist.
+.. include:: /includes/extracts/4.4-changes-transactions-upsert.rst
 
 .. include:: /includes/extracts/transactions-operations-write-concern.rst
 

source/reference/method/db.collection.bulkWrite.txt

@@ -488,8 +488,7 @@ Transactions
 
 .. include:: /includes/extracts/transactions-supported-operation.rst
 
-If run inside a transaction, the collection must already exist for
-insert and ``upsert: true`` operations.
+.. include:: /includes/extracts/4.4-changes-transactions-bulkWrite.rst
 
 .. include:: /includes/extracts/transactions-operations-write-concern.rst
 

source/reference/method/db.collection.insert.txt

@@ -125,9 +125,7 @@ Transactions
 
 .. include:: /includes/extracts/transactions-supported-operation.rst
 
-The collection must already exist. An insert operation that
-would result in the creation of a new collection are not allowed in a
-transaction.
+.. include:: /includes/extracts/4.4-changes-transactions-insert.rst
 
 .. include:: /includes/extracts/transactions-operations-write-concern.rst
 

source/reference/method/db.collection.insertMany.txt

@@ -158,9 +158,7 @@ Transactions
 
 .. include:: /includes/extracts/transactions-supported-operation.rst
 
-The collection must already exist. An insert operation that
-would result in the creation of a new collection are not allowed in a
-transaction.
+.. include:: /includes/extracts/4.4-changes-transactions-insert.rst
 
 .. include:: /includes/extracts/transactions-operations-write-concern.rst
 
@@ -404,5 +402,3 @@ This operation returns:
       },
       "errmsg" : "waiting for replication timed out"
    })
-
-

source/reference/method/db.collection.insertOne.txt

@@ -119,9 +119,7 @@ Transactions
 
 .. include:: /includes/extracts/transactions-supported-operation.rst
 
-The collection must already exist. An insert operation that
-would result in the creation of a new collection are not allowed in a
-transaction.
+.. include:: /includes/extracts/4.4-changes-transactions-insert.rst
 
 .. include:: /includes/extracts/transactions-operations-write-concern.rst
 

source/reference/method/db.collection.save.txt

@@ -112,8 +112,7 @@ Transactions
 
 .. include:: /includes/extracts/transactions-supported-operation.rst
 
-If the operation results in an insert, the collection must already
-exist.
+.. include:: /includes/extracts/4.4-changes-transactions-save.rst
 
 .. include:: /includes/extracts/transactions-operations-write-concern.rst
 

source/reference/method/db.collection.update.txt

@@ -417,11 +417,7 @@ Transactions
 Existing Collections and Transactions
 `````````````````````````````````````
 
-Inside a transaction, you can specify read/write operations on existing
-collections. If the |operation| results in an
-upsert, the collection must already exist.
-
-If the operation results in an upsert, the collection must already exist.
+.. include:: /includes/extracts/4.4-changes-transactions-upsert.rst
 
 Write Concerns and Transactions
 ````````````````````````````````

source/reference/method/db.collection.updateMany.txt

@@ -309,7 +309,7 @@ Transactions
 
 .. include:: /includes/extracts/transactions-supported-operation.rst
 
-If the operation results in an upsert, the collection must already exist.
+.. include:: /includes/extracts/4.4-changes-transactions-upsert.rst
 
 .. include:: /includes/extracts/transactions-operations-write-concern.rst
 

source/reference/method/db.collection.updateOne.txt

@@ -420,9 +420,7 @@ Transactions
 Existing Collections and Transactions
 `````````````````````````````````````
 
-Inside a transaction, you can specify read/write operations on existing
-collections. If the |operation| results in an
-upsert, the collection must already exist.
+.. include:: /includes/extracts/4.4-changes-transactions-upsert.rst
 
 Write Concerns and Transactions
 ````````````````````````````````