DOCS-11802,DOCS-11758,DOCS-11505: txn inmemory storage engine, create/drop collection before txn, data model updates, atomicity/isolation updates

Author: kay-kim

source/core/data-model-design.txt

@@ -48,24 +48,33 @@ as well as the ability to request and retrieve related data in a single
 database operation. Embedded data models make it possible to update
 related data in a single atomic write operation.
 
-However, embedding related data in documents may lead to situations
-where documents grow after creation. With the MMAPv1 storage engine,
-document growth can impact write performance and lead to data
-fragmentation. 
-
-In version 3.0.0, MongoDB uses :ref:`power-of-2-allocation` as the
-default allocation strategy for MMAPv1 in order to account for document
-growth, minimizing the likelihood of data fragmentation. See
-:ref:`power-of-2-allocation` for details. Furthermore, documents in
-MongoDB must be smaller than the :limit:`maximum BSON document size
-<BSON Document Size>`. For bulk binary data, consider :doc:`GridFS
+To access data within embedded documents, use :term:`dot notation` to
+"reach into" the embedded documents. See :ref:`query for data in arrays
+<read-operations-arrays>` and :ref:`query data in embedded documents
+<read-operations-embedded-documents>` for more examples on accessing
+data in arrays and embedded documents.
+
+Embedded Data Model and Document Size Limit
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Documents in MongoDB must be smaller than the :limit:`maximum BSON
+document size <BSON Document Size>`.
+
+For bulk binary data, consider :doc:`GridFS
 </core/gridfs>`.
 
-To interact with embedded documents, use :term:`dot notation` to "reach
-into" embedded documents. See :ref:`query for data in arrays
-<read-operations-arrays>` and :ref:`query data in embedded documents
-<read-operations-embedded-documents>` for more examples on accessing data in
-arrays and embedded documents.
+Embedded Data Model and Deprecated MMAPv1
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Embedding related data in documents may lead to situations where
+documents grow after creation. With the deprecated MMAPv1 storage
+engine, document growth can impact write performance and lead to data
+fragmentation.
+
+Starting in version 3.0.0, MongoDB uses :ref:`power-of-2-allocation` as
+the default allocation strategy for MMAPv1 in order to account for
+document growth, minimizing the likelihood of data fragmentation. See
+:ref:`power-of-2-allocation` for details.
 
 .. _data-modeling-referencing:
 

source/core/data-model-operations.txt

@@ -10,78 +10,47 @@ Operational Factors and Data Models
    :depth: 1
    :class: singlecol
 
-Modeling application data for MongoDB depends on both the data itself,
-as well as the characteristics of MongoDB itself. For example,
-different data models may allow applications to use more efficient
-queries, increase the throughput of insert and update operations, or
-distribute activity to a sharded cluster more effectively.
-
-These factors are *operational* or address requirements that arise
-outside of the application but impact the performance of MongoDB based
-applications. When developing a data model, analyze all of your
-application's :doc:`read and write operations </crud>` in conjunction
-with the following considerations.
+Modeling application data for MongoDB should consider various
+operational factors that impact the performance of MongoDB. For
+instance, different data models can allow for more efficent queries,
+increase the throughput of insert and update operations, or distribute
+activity to a sharded cluster more effectively.
 
-.. _data-model-document-growth:
-
-Document Growth
----------------
-
-.. versionchanged:: 3.0.0
-
-Some updates to documents can increase the size of documents. These
-updates include pushing elements to an array (i.e. :update:`$push`) and
-adding new fields to a document.
-
-When using the MMAPv1 storage engine, document growth can be a
-consideration for your data model. For MMAPv1, if the document size
-exceeds the allocated space for that document, MongoDB will relocate
-the document on disk. With MongoDB 3.0.0, however, the default use of
-the :ref:`power-of-2-allocation` minimizes the occurrences of such
-re-allocations as well as allows for the effective reuse of the freed
-record space.
-
-When using MMAPv1, if your applications require updates that will
-frequently cause document growth to exceeds the current power of 2
-allocation, you may want to refactor your data model to use references
-between data in distinct documents rather than a denormalized data
-model.
-
-You may also use a *pre-allocation* strategy to explicitly avoid
-document growth. Refer to the :ecosystem:`Pre-Aggregated Reports Use
-Case </use-cases/pre-aggregated-reports>` for an example of the
-*pre-allocation* approach to handling document growth.
-
-See :doc:`/core/mmapv1` for more information on MMAPv1.
+When developing a data model, analyze all of your application's
+:doc:`read and write operations </crud>` in conjunction with the
+following considerations.
 
 .. _data-model-atomicity:
 .. _data-modeling-atomicity:
 
 Atomicity
 ---------
 
-In MongoDB, operations are atomic at the :term:`document` level. No
-**single** write operation can change more than one document.
-Operations that modify more than a single document in a collection
-still operate on one document at a time. [#record-atomicity]_ Ensure
-that your application stores all fields with atomic dependency
-requirements in the same document. If the application can tolerate
-non-atomic updates for two pieces of data, you can store these data in
-separate documents.
-
-A data model that embeds related data in a single document
-facilitates these kinds of atomic operations. For data models that
-store references between related pieces of data, the application must
-issue separate read and write operations to retrieve and modify these
-related pieces of data.
+In MongoDB, a write operation is atomic on the level of a single
+document, even if the operation modifies multiple embedded documents
+*within* a single document. When a single write operation modifies
+multiple documents (e.g. :method:`db.collection.updateMany()`), the
+modification of each document is atomic, but the operation as a whole
+is not atomic.
+
+Embedded Data Model
+~~~~~~~~~~~~~~~~~~~
+
+The embedded data model combines all related data in a single document
+instead of normalizing across multiple documents and collections. This
+data model facilitates atomic operations.
 
 See :ref:`data-modeling-atomic-operation` for an example data model
 that provides atomic updates for a single document.
 
-.. [#record-atomicity] Document-level atomic operations include all
-   operations within a single MongoDB document record: operations that
-   affect multiple embedded documents within that single record are still
-   atomic.
+Multi-Document Transaction
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+For data models that store references between related
+pieces of data, the application must issue separate read and write
+operations to retrieve and modify these related pieces of data.
+
+.. include:: /includes/extracts/transactions-intro.rst
 
 Sharding
 --------
@@ -305,3 +274,36 @@ documents, consider :doc:`/core/capped-collections`. Capped collections
 provide *first-in-first-out* (FIFO) management of inserted documents
 and efficiently support operations that insert and read documents based
 on insertion order.
+
+.. _data-model-document-growth:
+
+Document Growth and MMAPv1
+--------------------------
+
+.. versionchanged:: 3.0.0
+
+Some updates to documents can increase the size of documents. These
+updates include pushing elements to an array (i.e. :update:`$push`) and
+adding new fields to a document.
+
+When using the deprecated MMAPv1 storage engine, document growth can be
+a consideration for your data model. For MMAPv1, if the document size
+exceeds the allocated space for that document, MongoDB will relocate
+the document on disk. Starting with MongoDB 3.0, however, the default
+use of the :ref:`power-of-2-allocation` minimizes the occurrences of
+such re-allocations as well as allows for the effective reuse of the
+freed record space.
+
+When using MMAPv1, if your applications require updates that will
+frequently cause document growth to exceeds the current power of 2
+allocation, you may want to refactor your data model to use references
+between data in distinct documents rather than a denormalized data
+model.
+
+You may also use a *pre-allocation* strategy to explicitly avoid
+document growth. Refer to the :ecosystem:`Pre-Aggregated Reports Use
+Case </use-cases/pre-aggregated-reports>` for an example of the
+*pre-allocation* approach to handling document growth.
+
+See :doc:`/core/mmapv1` for more information on MMAPv1.
+

source/core/data-modeling-introduction.txt

@@ -10,45 +10,46 @@ Data Modeling Introduction
    :depth: 1
    :class: singlecol
 
-Data in MongoDB has a *flexible schema*. Unlike SQL databases, where
-you must determine and declare a table's schema before inserting data,
-MongoDB's :term:`collections <collection>` do not enforce
-:term:`document` structure. This flexibility facilitates the mapping of
-documents to an entity or an object. Each document can match the data
-fields of the represented entity, even if the data has substantial
-variation. In practice, however, the documents in a collection share a
-similar structure.
-
 The key challenge in data modeling is balancing the needs of the
 application, the performance characteristics of the database engine, and
 the data retrieval patterns. When designing data models, always
 consider the application usage of the data (i.e. queries, updates, and
 processing of the data) as well as the inherent structure of the data
 itself.
 
-Document Structure
-------------------
+Flexible Schema
+---------------
 
-.. start-primer-excerpt
+Unlike SQL databases, where you must determine and declare a table's
+schema before inserting data, MongoDB's :term:`collections
+<collection>`, by default, does not require its :doc:`documents
+</core/document>` to have the same schema. That is:
 
-The key decision in designing data models for MongoDB applications
-revolves around the structure of documents and how the application
-represents relationships between data. There are two tools that allow
-applications to represent these relationships: *references* and
-*embedded documents*.
+- The documents in a single collection do not need to have the same set
+  of fields and the data type for a field can differ across documents
+  within a collection.
 
-References
-~~~~~~~~~~
+- To change the structure of the documents in a collection, such as add
+  new fields, remove existing fields, or change the field values to a
+  new type, update the documents to the new structure.
 
-References store the relationships between data by including
-links or *references* from one document to another. Applications can
-resolve these :doc:`references </reference/database-references>` to
-access the related data. Broadly, these are *normalized* data models.
+This flexibility facilitates the mapping of documents to an entity or
+an object. Each document can match the data fields of the represented
+entity, even if the document has substantial variation from other
+documents in the collection.
 
-.. include:: /images/data-model-normalized.rst
+In practice, however, the documents in a collection share a similar
+structure, and you can enforce :doc:`document validation rules
+</core/schema-validation>` for a collection during update and insert
+operations. See :doc:`/core/schema-validation` for details.
 
-See :ref:`data-modeling-referencing` for the strengths and weaknesses of
-using references.
+Document Structure
+------------------
+
+The key decision in designing data models for MongoDB applications
+revolves around the structure of documents and how the application
+represents relationships between data. MongoDB allows related data to
+be embedded within a single document.
 
 Embedded Data
 ~~~~~~~~~~~~~
@@ -62,42 +63,46 @@ database operation.
 
 .. include:: /images/data-model-denormalized.rst
 
+For many use cases in MongoDB, the denormalized data model is optimal.
+
 See :ref:`data-modeling-embedding` for the strengths and weaknesses of
 embedding documents.
 
-.. end-primer-excerpt
+References
+~~~~~~~~~~
+
+References store the relationships between data by including
+links or *references* from one document to another. Applications can
+resolve these :doc:`references </reference/database-references>` to
+access the related data. Broadly, these are *normalized* data models.
+
+.. include:: /images/data-model-normalized.rst
+
+See :ref:`data-modeling-referencing` for the strengths and weaknesses of
+using references.
 
 Atomicity of Write Operations
 -----------------------------
 
-In MongoDB, write operations are atomic at the :term:`document` level,
-and no single write operation can atomically affect more than one
-document or more than one collection. A denormalized data model with
-embedded data combines all related data for a represented entity in a
-single document. This facilitates atomic write operations since a
-single write operation can insert or update the data for an entity.
-Normalizing the data would split the data across multiple collections
-and would require multiple write operations that are not atomic
-collectively.
-
-However, schemas that facilitate atomic writes may limit ways that
-applications can use the data or may limit ways to modify applications.
-The :ref:`Atomicity Considerations <data-model-atomicity>`
-documentation describes the challenge of designing a schema that
-balances flexibility and atomicity.
-
-Document Growth
----------------
+Single Document Atomicity
+~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Some updates, such as pushing elements to an array or adding new
-fields, increase a :term:`document's <document>` size. 
+In MongoDB, a write operation is atomic on the level of a single
+document, even if the operation modifies multiple embedded documents
+*within* a single document.
+
+A denormalized data model with embedded data combines all related data
+in a single document instead of normalizing across multiple documents
+and collections. This data model facilitates atomic operations.
+
+.. include:: /includes/extracts/concurrent-operations-multi-document-writes.rst
 
-For the MMAPv1 storage engine, if the document size exceeds the
-allocated space for that document, MongoDB relocates the document on
-disk. When using the MMAPv1 storage engine, growth consideration can
-affect the decision to normalize or denormalize data. See
-:ref:`Document Growth Considerations <data-model-document-growth>` for
-more about planning for and managing document growth for MMAPv1.
+Multi-Document Transactions
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. include:: /includes/extracts/transactions-intro.rst
+
+.. seealso:: :ref:`Atomicity Considerations <data-model-atomicity>`
 
 Data Use and Performance
 ------------------------
@@ -110,3 +115,18 @@ adding indexes to support common queries can improve performance.
 
 See :doc:`/core/data-model-operations` for more information on these
 and other operational considerations that affect data model designs.
+
+Document Growth and MMAPv1
+--------------------------
+
+Some updates, such as pushing elements to an array or adding new
+fields, increase a :term:`document's <document>` size. 
+
+For the :doc:`deprecated MMAPv1 </core/mmapv1>` storage engine, if the
+document size exceeds the allocated space for that document, MongoDB
+relocates the document on disk. When using the deprecated MMAPv1
+storage engine, growth consideration can affect the decision to
+normalize or denormalize data. See :ref:`Document Growth Considerations
+<data-model-document-growth>` for more about planning for and managing
+document growth for MMAPv1.
+

source/core/inmemory.txt

@@ -134,6 +134,12 @@ Write operations that specify a write concern :writeconcern:`journaled
 shuts down, either as result of the :dbcommand:`shutdown` command or
 due to a system error, recovery of in-memory data is impossible.
 
+Transactions
+------------
+
+Multi-document transactions are not available for deployments with
+members that use in-memory storage engine.
+
 Deployment Architectures
 ------------------------