You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<li><code>memory</code> - memory used by the queue process, accounts message metadata (at least 720 bytes per message), does not account for message payloads over 64 bytes</li>
697
+
<li><code>message_bytes_ram</code> - memory used by the message payloads, regardless of size</li>
698
+
</ul>
699
+
700
+
<p>
701
+
If messages are small, message metadata can use more memory than the message payload.
702
+
10,000 messages with 1 byte of payload will use 10KB of <code>message_bytes_ram</code> (payload) & 7MB of <code>memory</code> (metadata).
703
+
</p>
704
+
705
+
<p>
706
+
If message payloads are large, they will not be reflected in the queue process memory.
707
+
10,000 messages with 100 KB of payload will use 976MB of <code>message_bytes_ram</code> (payload) & 7MB of <code>memory</code> (metadata).
708
+
</p>
709
+
</doc:subsection>
710
+
711
+
<doc:subsectionname="queue-memory-usage">
712
+
<doc:heading>Why does the queue memory grow and shrink when publishing/consuming?</doc:heading>
713
+
714
+
<p>
715
+
Erlang uses <ahref="https://www.erlang-solutions.com/blog/erlang-19-0-garbage-collector.html"target="_blank">generational garbage collection</a> for each Erlang process.
716
+
Garbage collection is done per queue, independently of all other Erlang processes.
717
+
</p>
718
+
719
+
<p>
720
+
When garbage collection runs, it will copy used process memory before deallocating unused memory.
721
+
This can can lead to the queue process using up to twice as much memory during garbage collection, as shown here (queue contains a lot of messages):
722
+
723
+
<imgsrc="img/memory/queue-memory-usage-spikes.png"alt="Queue under load memory usage"title="Queue under load memory usage" />
724
+
</p>
725
+
</doc:subsection>
726
+
727
+
<doc:subsectionname="queue-memory-usage">
728
+
<doc:heading>Can queue memory growth during garbage collection be dangerous?</doc:heading>
729
+
<p>
730
+
If Erlang VM tries to allocate more memory than is available, the VM itself will either crash or be killed by the OOM killer.
731
+
When the Erlang VM crashes, RabbitMQ will lose all non-persistent data.
732
+
</p>
733
+
734
+
<p>
735
+
High memory watermark blocks publishers and prevents new messages from being enqueued.
736
+
Since garbage collection can double the memory used by a queue, it is unsafe to set the high memory watermark above <code>0.5</code>.
737
+
The default high memory watermark is set to <code>0.4</code> since this is safer as not all memory is used by queues.
738
+
This is entirely workload specific, which differs across RabbitMQ deployments.
739
+
</p>
740
+
741
+
<p>
742
+
We recommend many queues so that memory allocation / garbage collection is spread across many Erlang processes.
743
+
</p>
744
+
745
+
<p>
746
+
If the messages in a queue take up a lot of memory, we recommend lazy queues so that they are stored on disk as soon as possible and not kept in memory longer than is necessary.
0 commit comments