Skip to content

Commit f0ffe99

Browse files
authored
Merge pull request #455 from rabbitmq/rabbitmq-website-queue-memory
FAQ about queue memory use.
2 parents 11986e1 + 676e7e7 commit f0ffe99

File tree

5 files changed

+120
-5
lines changed

5 files changed

+120
-5
lines changed

site/admin-guide.xml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -181,6 +181,9 @@ limitations under the License.
181181
<h2>Message Store and Resource Management</h2>
182182

183183
<ul>
184+
<li>
185+
<a href="memory-use.html">Memory Usage</a>
186+
</li>
184187
<li>
185188
<a href="memory.html">Memory Management</a>
186189
</li>

site/documentation.xml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -248,6 +248,9 @@ limitations under the License.
248248
Message Store and Resource Management:
249249

250250
<ul>
251+
<li>
252+
<a href="memory-use.html">Memory Usage</a>
253+
</li>
251254
<li>
252255
<a href="memory.html">Memory Management</a>
253256
</li>
171 KB
Loading

site/memory-use.xml

Lines changed: 113 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -365,8 +365,10 @@ limitations under the License.
365365
</p>
366366

367367
<p>
368-
<pre class="sourcecode ini">
369-
curl -s -u guest:guest -X GET http://127.0.0.1:15672/api/nodes/rabbit@mercurio/memory | python -m json.tool
368+
<pre class="sourcecode json">
369+
curl -s -u guest:guest http://127.0.0.1:15672/api/nodes/rabbit@mercurio/memory |
370+
python -m json.tool
371+
370372
{
371373
"memory": {
372374
"atom": 1041593,
@@ -401,8 +403,10 @@ curl -s -u guest:guest -X GET http://127.0.0.1:15672/api/nodes/rabbit@mercurio/m
401403
</p>
402404

403405
<p>
404-
<pre class="sourcecode ini">
405-
curl -s -u guest:guest -X GET http://127.0.0.1:15672/api/nodes/rabbit@mercurio/memory/relative | python -m json.tool
406+
<pre class="sourcecode json">
407+
curl -s -u guest:guest http://127.0.0.1:15672/api/nodes/rabbit@mercurio/memory/relative |
408+
python -m json.tool
409+
406410
{
407411
"memory": {
408412
"allocated_unused": 32,
@@ -638,5 +642,110 @@ curl -s -u guest:guest -X GET http://127.0.0.1:15672/api/nodes/rabbit@mercurio/m
638642
documentation.
639643
</p>
640644
</doc:section>
645+
646+
<doc:section name="queue-memory">
647+
<doc:heading>Queue Memory</doc:heading>
648+
<doc:subsection name="message-memory-usage">
649+
<doc:heading>How much memory does a message use?</doc:heading>
650+
651+
<p>A message has multiple parts that use up memory:</p>
652+
653+
<ul class="plain">
654+
<li>Payload - >= 1 byte - variable size, typically few hundred bytes to a few hundred kilobytes</li>
655+
<li>Protocol attributes - >= 0 bytes - variable size, contains headers, priority, timestamp, reply to, etc.</li>
656+
<li>RabbitMQ metadata - >= 720 bytes - variable size, contains exchange, routing keys, message properties, persistence, redelivery status, etc.</li>
657+
<li>RabbitMQ message ordering structure - 16 bytes</li>
658+
</ul>
659+
660+
<p>Messages with a 1KB payload will use up 2KB of memory once attributes and metadata is factored in.</p>
661+
662+
<p>Some messages can be stored on disk, but still have their metadata kept in memory.</p>
663+
</doc:subsection>
664+
665+
<doc:subsection name="queue-memory-usage">
666+
<doc:heading>How much memory does a queue use?</doc:heading>
667+
668+
<p>A message has multiple parts that use up memory:</p>
669+
670+
<p>A queue is an Erlang process. If a queue is mirrored, each mirror is a separate Erlang process.</p>
671+
672+
<p>
673+
Since a queues master is a single Erlang process, message ordering can be guaranteed.
674+
Multiple queues means multiple Erlang processes which get an even amount of CPU time.
675+
This ensures that no queue can block other queues.
676+
</p>
677+
678+
<p>
679+
The memory use of a single queue can be obtained via the HTTP API:
680+
681+
<pre class="sourcecode json">
682+
curl -s -u guest:guest http://127.0.0.1:15672/api/queues/%2f/queue-name |
683+
python -m json.tool
684+
685+
{
686+
..
687+
"memory": 97921904,
688+
...
689+
"message_bytes_ram": 2153429941,
690+
...
691+
}
692+
</pre>
693+
</p>
694+
695+
<ul class="plain">
696+
<li><code>memory</code> - memory used by the queue process, accounts message metadata (at least 720 bytes per message), does not account for message payloads over 64 bytes</li>
697+
<li><code>message_bytes_ram</code> - memory used by the message payloads, regardless of size</li>
698+
</ul>
699+
700+
<p>
701+
If messages are small, message metadata can use more memory than the message payload.
702+
10,000 messages with 1 byte of payload will use 10KB of <code>message_bytes_ram</code> (payload) &amp; 7MB of <code>memory</code> (metadata).
703+
</p>
704+
705+
<p>
706+
If message payloads are large, they will not be reflected in the queue process memory.
707+
10,000 messages with 100 KB of payload will use 976MB of <code>message_bytes_ram</code> (payload) &amp; 7MB of <code>memory</code> (metadata).
708+
</p>
709+
</doc:subsection>
710+
711+
<doc:subsection name="queue-memory-usage">
712+
<doc:heading>Why does the queue memory grow and shrink when publishing/consuming?</doc:heading>
713+
714+
<p>
715+
Erlang uses <a href="https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html" target="_blank">generational garbage collection</a> for each Erlang process.
716+
Garbage collection is done per queue, independently of all other Erlang processes.
717+
</p>
718+
719+
<p>
720+
When garbage collection runs, it will copy used process memory before deallocating unused memory.
721+
This can can lead to the queue process using up to twice as much memory during garbage collection, as shown here (queue contains a lot of messages):
722+
723+
<img src="img/memory/queue-memory-usage-spikes.png" alt="Queue under load memory usage" title="Queue under load memory usage" />
724+
</p>
725+
</doc:subsection>
726+
727+
<doc:subsection name="queue-memory-usage">
728+
<doc:heading>Can queue memory growth during garbage collection be dangerous?</doc:heading>
729+
<p>
730+
If Erlang VM tries to allocate more memory than is available, the VM itself will either crash or be killed by the OOM killer.
731+
When the Erlang VM crashes, RabbitMQ will lose all non-persistent data.
732+
</p>
733+
734+
<p>
735+
High memory watermark blocks publishers and prevents new messages from being enqueued.
736+
Since garbage collection can double the memory used by a queue, it is unsafe to set the high memory watermark above <code>0.5</code>.
737+
The default high memory watermark is set to <code>0.4</code> since this is safer as not all memory is used by queues.
738+
This is entirely workload specific, which differs across RabbitMQ deployments.
739+
</p>
740+
741+
<p>
742+
We recommend many queues so that memory allocation / garbage collection is spread across many Erlang processes.
743+
</p>
744+
745+
<p>
746+
If the messages in a queue take up a lot of memory, we recommend lazy queues so that they are stored on disk as soon as possible and not kept in memory longer than is necessary.
747+
</p>
748+
</doc:subsection>
749+
</doc:section>
641750
</body>
642751
</html>

site/queues.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -195,7 +195,7 @@ or when a certain amount of time passes (fraction of a second).
195195
[Lazy queues](/lazy-queues.html) page messages out to disk more aggressively
196196
regardless of their persistence property.
197197

198-
See [Memory Usage](/memory-use.html), [Alarms](/alarms.html),
198+
See [Memory Usage](/memory-use.html), [Alarms](/alarms.html)
199199
[Memory Alarms](http://localhost:8191/memory.html), [Free Disk Space Alarms](/disk-alarms.html),
200200
[Production Checklist](/production-checklist.html), and [Message Store Configuration](/persistence-conf.html)
201201
guide for details.

0 commit comments

Comments
 (0)