DOCS-687 new Concurrency FAQ

bin/builder_data.py

@@ -83,6 +83,7 @@
      ('$(rst-include)/table-sql-to-mongo-select-examples', 'sql'),
      ('$(rst-include)/table-sql-to-mongo-update-examples', 'sql'),
      ('$(rst-include)/table-sql-to-mongo-delete-examples', 'sql'),
+     ('$(rst-include)/table-lock-behavior-per-operation', 'concurrency'),
 ]
 
 sphinx = [

source/faq.txt

@@ -7,6 +7,7 @@ Frequently Asked Questions
 
    /faq/fundamentals
    /faq/developers
+   /faq/concurrency
    /faq/sharding
    /faq/replica-sets
    /faq/storage

source/faq/concurrency.txt

@@ -0,0 +1,181 @@
+================
+FAQ: Concurrency
+================
+
+.. default-domain:: mongodb
+
+MongoDB allows multiple clients to read and write a single corpus of
+data using a locking system to ensure that all clients receive a
+consistent view of the data *and* to prevent multiple applications from
+modifying the exact same pieces of data at the same time. Locks help
+guarantee that all writes to a single document occur occur either in
+full or not at all.
+
+.. contents:: Frequently Asked Questions:
+   :backlinks: none
+   :local:
+
+.. _faq-concurrency-locking:
+
+What type of locking does MongoDB use?
+--------------------------------------
+
+MongoDB uses a readers-writer lock that allows concurrent reads access
+to a database but gives exclusive access to a single write operation.
+
+When a read holds the lock, other reads can share it. However, when
+a write holds the lock, the write holds the lock exclusively. No other
+read nor write can share the lock.
+
+Locks are "writer greedy," which means writes have preference over
+reads. When both a read and write are waiting for a lock, MongoDB
+grants the lock to the write.
+
+.. versionchanged:: 2.2
+   Beginning with version 2.2, MongoDB implements locks on a
+   per-database basis. If you have six databases and one takes a write
+   lock, the other five are still available for reads. In versions prior
+   to 2.2, MongoDB implements locks per :program:`mongod` instance.
+
+For more information on locks, see the :data:`locks` documentation.
+
+How do I view lock status?
+--------------------------
+
+To view what operations are running and what each operation has locked,
+use :method:`currentOp() <db.currentOp()`>.
+
+To view lock status, use :method:`serverStatus() <db.serverStatus()>`.
+Optionally, you also can use the :doc:`mongotop </reference/mongotop>`
+utility, the :doc:`mongostat </reference/mongostat>` utility, or the
+`MongoDB Monitoring Service (MMS) <http://mms.10gen.com/>`_.
+
+To terminate an operation, use :method:`killOp() <db.killOp()>`.
+
+Does a read or write ever yield the lock?
+-----------------------------------------
+
+A read or write operation will yield a lock when MongoDB attempts to read data
+that is not yet in memory. When MongoDB attempts to read data not in memory,
+a :term:`page fault` occurs.
+
+.. versionchanged:: 2.2
+   MongoDB tracks the contents of memory and predicts whether data is
+   available before performing a read. If data is predicted to be
+   unavailable, a read operation yields its lock and requests that
+   MongoDB download the data to memory. Once data is available in
+   memory, the read retakes the lock and completes the operation.
+
+Which operations lock the database?
+-----------------------------------
+
+The following table lists common database operations and the types of
+locks they use.
+
+.. todo In the table below (in the include), the issue of blocked
+   JavaScript might no longer apply in version 2.4, which will use V8.
+
+.. include:: /includes/table-lock-behavior-per-operation.rst
+
+Which administrative commands lock the database?
+------------------------------------------------
+
+Certain administrative commands can exclusively lock the database for
+extended periods of time. On very large databases, consider taking the
+the :program:`mongod` instance offline so that clients are not affected.
+For example, if a :program:`mongod` is part of a :term:`replica set`,
+take the :program:`mongod` offline and let other members of the set
+service load while maintenance is in progress.
+
+The following administrative commands can exclusively lock the database for
+extended periods:
+
+- :method:`db.collection.ensureIndex()`, when issued
+  *without* setting ``background`` to ``true``
+- :dbcommand:`reIndex`
+- :dbcommand:`compact`
+- :method:`db.repairDatabase()`
+- :method:`db.createCollection()`, when creating a very large (many
+  gigabytes) capped collection
+- :method:`db.collection.validate()`
+- :method:`db.copyDatabase()`. This command can lock all databases. See
+  :ref:`faq-concurrency-lock-multiple-dbs`.
+
+The following administrative commands lock the database but are fast and
+do not excessively block the system:
+
+- :method:`db.collection.dropIndex()`
+- :method:`db.collection.getLastError()`
+- :method:`db.isMaster()`
+- :method:`rs.status()` (and :dbcommand:`replSetGetStatus`)
+- :method:`db.serverStatus()`
+- :method:`db.auth()`
+- :method:`db.addUser()`
+
+.. _faq-concurrency-lock-multiple-dbs:
+
+Does a MongoDB operation ever lock more than one database?
+----------------------------------------------------------
+
+The following MongoDB operations lock multiple databases:
+
+- :method:`db.copyDatabase()` performs a global lock.
+
+- Journaling, which is an internal operation, locks all databases for
+  short intervals. The journal is shared by all databases.
+
+- :ref:`User authentication <security-authentication>` locks the
+  ``admin`` database as well as the database the user is accessing.
+
+- Writes to a replica set's :term:`primary` lock both the database
+  receiving the writes and the ``local`` database, the latter in order
+  to record the writes to the primary's :term:`oplog`.
+
+How does sharding affect concurrency?
+-------------------------------------
+
+:term:`Sharding <sharding>` improves concurrency by distributing
+collections over multiple :program:`mongod` instances, allowing shard
+servers (i.e. :program:`mongos` processes) to perform any number of
+operations concurrently to the various downstream :program:`mongod`
+instances.
+
+Each :program:`mongod` instance is independent of the others in the
+shard cluster and uses the MongoDB :ref:`readers-writer lock
+<faq-concurrency-locking>`). The operations on one :program:`mongod`
+instance do not block the operations on any others.
+
+.. _faq-concurrency-replication:
+
+How does concurrency affect a replica set primary?
+--------------------------------------------------
+
+In :term:`replication`, when MongoDB writes to a collection on the
+:term:`primary`, MongoDB also writes to the primary's :term:`oplog`.
+Therefore, MongoDB must lock both the collection's database and the
+oplog's database. The oplog is located in the ``local`` database. Both
+must be locked at same time to keep the operation atomic. It is an
+all-or-nothing operation.
+
+How does concurrency affect secondaries?
+----------------------------------------
+
+In :term:`replication`, MongoDB does not apply writes serially to
+:term:`secondaries <secondary>`. Secondaries collect oplog entries in
+batches and then apply those batches in parallel. Secondaries do not
+allow reads while applying the batch writes.
+
+A primary might apply multiple writes to different databases
+simultaneously. When the primary finishes each write, it assigns each write
+to a place in the oplog, giving the writes a specific order.
+
+The secondaries apply the writes in the order in the oplog.
+
+MongoDB can apply several writes in parallel on replica set secondaries.
+This is done in two phases:
+
+- A prefetch phase occurs during a read lock. During this phase other
+  clients may execute queries.
+
+- A thread pool using write locks applies a batch of writes in a
+  coordinated write phase.

source/faq/developers.txt

@@ -342,10 +342,12 @@ See the "`PHP MongoDB Driver Security Notes
 <http://us.php.net/manual/en/mongo.security.php>`_" page in the PHP
 driver documentation for more information
 
+.. _faq-dev-concurrency:
+
 How does MongoDB provide concurrency?
 -------------------------------------
 
-MongoDB implements a server-wide reader-writer lock. This means that
+MongoDB implements a readers-writer lock. This means that
 at any one time, only one client may be writing or any number
 of clients may be reading, but that reading and writing cannot
 occur simultaneously.
@@ -355,16 +357,7 @@ applies to a single :program:`mongod` instance or :term:`primary`
 instance. In a sharded cluster, locks apply to each individual shard,
 not to the whole cluster.
 
-A more granular approach to locking will appear in MongoDB v2.2. For
-now, several yielding optimizations exist to mitigate the coarseness
-of the lock. These include:
-
-- Yielding on long operations. Queries and updates that operate on
-  multiple document may yield to writers
-
-- Yielding on page faults. If an update or query is likely to trigger
-  a page fault, then the operation will yield to keep from blocking
-  other clients for the duration of the page fault.
+For more information, see :doc:`/faq/concurrency`.
 
 What is the compare order for BSON types?
 -----------------------------------------
@@ -427,8 +420,6 @@ for additional information.
    - The :source:`jsobj.h <src/mongo/db/jsobj.h>` source file for the
      definition of ``MinKey`` and ``MaxKey``.
 
-.. _faq-developers:
-
 .. _faq-restrictions-on-collection-names:
 
 Are there any restrictions on the names of Collections?

source/includes/table-lock-behavior-per-operation.yaml

@@ -0,0 +1,46 @@
+# table structure. all content symbolic.
+section: layout
+header: [ meta.header1, meta.header2 ]
+rows:
+  - 1: [ content.op1, content.lock1 ]
+  - 2: [ content.op2, content.lock2 ]
+  - 3: [ content.op3, content.lock3 ]
+  - 4: [ content.op4, content.lock4 ]
+  - 5: [ content.op5, content.lock5 ]
+  - 6: [ content.op6, content.lock6 ]
+  - 7: [ content.op7, content.lock7 ]
+  - 8: [ content.op8, content.lock8 ]
+  - 9: [ content.op9, content.lock9 ]
+---
+# table metadata, as meta.<key>
+section: meta
+header1: "Operation"
+header2: "Lock Type"
+---
+# table content, as content.<key>
+section: content
+op1: Issue a query
+lock1: Read lock
+op2: Get more data from a :term:`cursor`
+lock2: Read lock
+op3: Insert data
+lock3: Write lock
+op4: Remove data
+lock4: Write lock
+op5: Update data
+lock5: Write lock
+op6: :term:`Map-reduce <map-reduce>`
+lock6: |
+       Read lock and write lock, unless operations are specified as
+       non-atomic. Portions of map-reduce jobs can run concurrently.
+op7: Create an index
+lock7: |
+       Building an index in the foreground, which is the default, locks
+       the database for extended periods of time.
+op8: :method:`db.eval()`
+lock8: |
+       Write lock or no lock. If this is used without the ``nolock``
+       option, all JavaScript is blocked.
+op9: :method:`aggregate() <db.collection.aggregate()>`
+lock9: Read lock
+...