DOCS-5211: Split out Analyzing MongoDB performance article

Author: i80and

config/redirects.yaml

@@ -211,8 +211,15 @@ code: 301
 outputs:
   - 'before-v2.6'
 ---
+from: '/administration/analyzing-mongodb-performance'
+to: '/administration/optimization'
+type: 'redirect'
+code: 301
+outputs:
+  - 'before-v2.6'
+---
 # redirected in 3.0 to getting started
-# temp -- we should fix giza so that 
+# temp -- we should fix giza so that
 # we can use the external field to redirect with
 # the after-xxx outputs
 from: '/tutorial/getting-started'
@@ -225,7 +232,7 @@ outputs:
   - { 'v3.0': "http://docs.mongodb.org/getting-started" }
 ---
 # redirected in 3.0 to getting started
-# temp -- we should fix giza so that 
+# temp -- we should fix giza so that
 # we can use the external field to redirect with
 # the after-xxx outputs
 from: '/tutorial/generate-test-data'

source/administration/analyzing-mongodb-performance.txt

@@ -0,0 +1,249 @@
+=============================
+Analyzing MongoDB Performance
+=============================
+
+.. default-domain:: mongodb
+
+As you develop and operate applications with MongoDB, you may need to
+analyze the performance of the application and its database.
+When you encounter degraded performance, it is often a function of database
+access strategies, hardware availability, and the number of open database
+connections.
+
+Some users may experience performance limitations as a result of inadequate
+or inappropriate indexing strategies, or as a consequence of poor schema
+design patterns. :ref:`analyzing-performance-locks` discusses how these can
+impact MongoDB's internal locking.
+
+Performance issues may indicate that the database is operating at capacity
+and that it is time to add additional capacity to the database. In particular,
+the application's :term:`working set` should fit in the available physical
+memory. See :ref:`analyzing-memory-mmapv1` for more information on the working
+set.
+
+In some cases performance issues may be temporary and related to
+abnormal traffic load. As discussed in :ref:`number-of-connections`, scaling
+can help relax excessive traffic.
+
+:ref:`database-profiling` can help you to understand what operations are causing
+degradation.
+
+.. _analyzing-performance-locks:
+
+Locking Performance
+~~~~~~~~~~~~~~~~~~~
+
+MongoDB uses a locking system to ensure data set consistency. If
+certain operations are long-running or a queue forms, performance
+will degrade as requests and operations wait for the lock.
+
+Lock-related slowdowns can be intermittent. To see if the lock has been
+affecting your performance, look to the data in the
+:ref:`globalLock` section of the :dbcommand:`serverStatus` output.
+
+If :data:`globalLock.currentQueue.total
+<serverStatus.globalLock.currentQueue.total>` is consistently high,
+then there is a chance that a large number of requests are waiting for
+a lock. This indicates a possible concurrency issue that may be affecting
+performance.
+
+If :data:`globalLock.totalTime <serverStatus.globalLock.totalTime>` is
+high relative to :data:`~serverStatus.uptime`, the database has
+existed in a lock state for a significant amount of time.
+
+If :data:`globalLock.ratio <serverStatus.globalLock.ratio>` is also high,
+MongoDB has likely been processing a large number of long running
+queries.
+
+Long queries can result from ineffective use of indexes;
+non-optimal schema design; poor query structure; system architecture issues; or
+insufficient RAM resulting in
+:ref:`page faults <administration-monitoring-page-faults>` and disk reads.
+
+.. _analyzing-memory-mmapv1:
+
+Memory and the MMAPv1 Storage Engine
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Memory Use
+``````````
+
+With the :ref:`MMAPv1 <storage-mmapv1>` storage engine, MongoDB uses
+memory-mapped files to store data. Given a data set of sufficient size,
+the :program:`mongod` process will allocate all available memory on the system
+for its use.
+
+While this is intentional and aids performance, the memory mapped files make it
+difficult to determine if the amount of RAM is sufficient for the data set.
+
+The :ref:`memory usage statuses <memory-status>` metrics of the
+:dbcommand:`serverStatus` output can provide insight into MongoDB's
+memory use.
+
+The :data:`mem.resident <serverStatus.mem.resident>` field provides the
+amount of resident memory in use. If this exceeds the amount of system
+memory *and* there is a significant amount of data on disk that isn't in RAM,
+you may have exceeded the capacity of your system.
+
+You can inspect :data:`mem.mapped <serverStatus.mem.mapped>` to check the
+amount of mapped memory that :program:`mongod` is using. If this value is
+greater than the amount of system memory, some operations will require a
+:term:`page faults <page fault>` to read data from disk.
+
+
+.. _administration-monitoring-page-faults:
+
+Page Faults
+```````````
+
+.. include:: /includes/fact-page-fault.rst
+
+MongoDB reports its triggered page faults as the total number of
+:term:`page faults <page fault>` in one second. To check for page faults, see
+the :data:`extra_info.page_faults <serverStatus.extra_info.page_faults>` value
+in the :dbcommand:`serverStatus` output.
+
+Rapid increases in the MongoDB page fault counter may indicate that the server
+has too little physical memory. Page faults also can occur while accessing
+large data sets or scanning an entire collection.
+
+A single page fault completes quickly and is not problematic. However, in
+aggregate, large volumes of page faults typically indicate that MongoDB
+is reading too much data from disk.
+
+MongoDB can often "yield" read locks after a page fault, allowing other database
+processes to read while :program:`mongod` loads the next page into memory.
+Yielding the read lock following a page fault improves concurrency, and also
+improves overall throughput in high volume systems.
+
+Increasing the amount of RAM accessible to MongoDB may help reduce the
+frequency of page faults. If this is not possible, you may want to consider
+deploying a :term:`sharded cluster` or adding :term:`shards <shard>`
+to your deployment to distribute load among :program:`mongod` instances.
+
+See :ref:`faq-storage-page-faults` for more information.
+
+.. _number-of-connections:
+
+Number of Connections
+~~~~~~~~~~~~~~~~~~~~~
+
+In some cases, the number of connections between the applications and the
+database can overwhelm the ability of the server to handle requests. The
+following fields in the :dbcommand:`serverStatus` document can provide insight:
+
+- :data:`globalLock.activeClients
+  <serverStatus.globalLock.activeClients>` contains a counter of the
+  total number of clients with active operations in progress or
+  queued.
+
+- :data:`~serverStatus.connections` is a container for the following
+  two fields:
+
+  - :data:`~serverStatus.connections.current` the total number of
+    current clients that connect to the database instance.
+
+  - :data:`~serverStatus.connections.available` the total number of
+    unused connections available for new clients.
+
+If there are numerous concurrent application requests, the database may have
+trouble keeping up with demand. If this is the case, then you will need to
+increase the capacity of your deployment.
+
+For read-heavy applications, increase the size of your :term:`replica set` and
+distribute read operations to :term:`secondary` members.
+
+For write-heavy applications, deploy :term:`sharding` and add one or more
+:term:`shards <shard>` to a :term:`sharded cluster` to distribute load among
+:program:`mongod` instances.
+
+Spikes in the number of connections can also be the result of
+application or driver errors. All of the officially supported MongoDB
+drivers implement connection pooling, which allows clients to use and
+reuse connections more efficiently. Extremely high numbers of
+connections, particularly without corresponding workload is often
+indicative of a driver or other configuration error.
+
+Unless constrained by system-wide limits, MongoDB has no limit on
+incoming connections. On Unix-based systems, you can modify system limits
+using the ``ulimit`` command, or by editing your system's
+``/etc/sysctl`` file. See :doc:`/reference/ulimit` for more
+information.
+
+.. _database-profiling:
+
+Database Profiling
+~~~~~~~~~~~~~~~~~~
+
+MongoDB's "Profiler" is a database profiling system that can help identify
+inefficient queries and operations.
+
+The following profiling levels are available:
+
+.. list-table::
+   :header-rows: 1
+
+   * - **Level**
+
+     - **Setting**
+
+   * - 0
+
+     - Off. No profiling
+
+   * - 1
+
+     - On. Only includes *"slow"* operations
+
+   * - 2
+
+     - On. Includes *all* operations
+
+Enable the profiler by setting the
+:dbcommand:`profile` value using the following command in the
+:program:`mongo` shell:
+
+.. code-block:: javascript
+
+   db.setProfilingLevel(1)
+
+The :setting:`~operationProfiling.slowOpThresholdMs` setting defines what constitutes a "slow"
+operation. To set the threshold above which the profiler considers
+operations "slow" (and thus, included in the level ``1`` profiling
+data), you can configure :setting:`~operationProfiling.slowOpThresholdMs` at runtime as an argument to
+the :method:`db.setProfilingLevel()` operation.
+
+.. see:: The documentation of :method:`db.setProfilingLevel()` for more
+   information.
+
+By default, :program:`mongod` records all "slow" queries to its
+:setting:`log <logpath>`, as defined by :setting:`~operationProfiling.slowOpThresholdMs`.
+
+.. note::
+
+   Because the database profiler can negatively impact
+   performance, only enable profiling for strategic intervals and as
+   minimally as possible on production systems.
+
+   You may enable profiling on a per-:program:`mongod` basis. This
+   setting will not propagate across a :term:`replica set` or
+   :term:`sharded cluster`.
+
+You can view the output of the profiler in the ``system.profile``
+collection of your database by issuing the ``show profile`` command in
+the :program:`mongo` shell, or with the following operation:
+
+.. code-block:: javascript
+
+   db.system.profile.find( { millis : { $gt : 100 } } )
+
+This returns all operations that lasted longer than 100 milliseconds.
+Ensure that the value specified here (``100``, in this example) is above the
+:setting:`~operationProfiling.slowOpThresholdMs` threshold.
+
+You must use the :operator:`$query` operator to access the ``query``
+field of documents within ``system.profile``.
+
+.. seealso:: :doc:`/administration/optimization` addresses strategies
+   that may improve the performance of your database queries and
+   operations.