Skip to content

Commit d96dcfd

Browse files
committed
DOCS-829 port map reduce concurrency section
1 parent 911d5a3 commit d96dcfd

File tree

2 files changed

+52
-6
lines changed

2 files changed

+52
-6
lines changed

source/applications/map-reduce.txt

Lines changed: 49 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -169,8 +169,9 @@ Run the first map-reduce operation as follows:
169169
Subsequent Incremental Map-Reduce
170170
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
171171

172-
Later when, the ``sessions`` collection grows, by adding the following
173-
documents, you can run additional map-reduce operations:
172+
Later as the ``sessions`` collection grows, you can run additional
173+
map-reduce operations. For example, add new documents to the
174+
``sessions`` collection:
174175

175176
.. code-block:: javascript
176177

@@ -208,6 +209,50 @@ periodically with the same target collection name without affecting
208209
the intermediate states. Use this mode when
209210
generating statistical output collections on a regular basis.
210211

212+
.. _map-reduce-concurrency:
213+
214+
Concurrency
215+
-----------
216+
217+
The map-reduce operation is composed of many tasks, including:
218+
219+
- reads from the input collection,
220+
221+
- executions of the ``map`` function,
222+
223+
- executions of the ``reduce`` function,
224+
225+
- writes to the output collection.
226+
227+
These various tasks take the following locks:
228+
229+
- The reads from the input collection takes a read lock that is yielded
230+
every 100 documents.
231+
232+
- The JavaScript code (i.e. ``map``, ``reduce``, ``finalize``
233+
functions) is executed in a single thread, taking a JavaScript lock;
234+
however, most JavaScript tasks in map-reduce are very short and
235+
yields the lock frequently.
236+
237+
- The insert into the temporary collection takes a write lock for a
238+
single write.
239+
240+
If the final collection does not exist, the creation of the final
241+
collection the output actions takes a write lock. If the final
242+
collection exists, then the output actions (i.e. ``merge``,
243+
``replace``, ``reduce``) take a write lock.
244+
245+
Although single-threaded, the map-reduce tasks interleave and appear to
246+
run in parallel.
247+
248+
.. note::
249+
250+
The final write lock during post-processing makes the results appear
251+
atomically. However, output actions ``merge`` and ``reduce`` may
252+
take minutes to process. For the ``merge`` and ``reduce``, the
253+
``nonAtomic`` flag is available. See the
254+
:method:`db.collection.mapReduce()` reference for more information.
255+
211256
.. _map-reduce-sharded-cluster:
212257

213258
Sharded Cluster
@@ -271,10 +316,10 @@ In MongoDB 2.0:
271316

272317
.. warning::
273318

274-
For best results only use the sharded output options for
319+
For best results, only use the sharded output options for
275320
:dbcommand:`mapReduce` in version 2.2 or later.
276321

277-
Troubleshooting Map Reduce Operations
322+
Troubleshooting Map-Reduce Operations
278323
-------------------------------------
279324

280325
You can troubleshoot the ``map`` function and the ``reduce`` function

source/includes/parameters-map-reduce.rst

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -182,12 +182,13 @@
182182
.. versionadded:: 2.1
183183

184184
Optional. Specify output operation as non-atomic and is
185-
valid *only* for ``merge`` and ``reduce`` output modes.
185+
valid *only* for ``merge`` and ``reduce`` output modes which
186+
may take minutes to execute.
186187

187188
If ``nonAtomic`` is ``true``, the post-processing step will
188189
prevent MongoDB from locking the database; however, other
189190
clients will be able to read intermediate states of the
190-
output database. Otherwise the map reduce operation must
191+
output collection. Otherwise the map reduce operation must
191192
lock the database during post-processing.
192193

193194
- **Output inline**. Perform the map-reduce operation in memory

0 commit comments

Comments
 (0)