Disable background GC by default; bump credit flow defaults #1098
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Quite a bit of evidence of different kinds of workloads suggest that background GC does not contribute meaningfully to reducing node RAM usage on most of them. It does, however, on each run produce a massive spike in VM and CPU context switches, causing latency spikes. Some users report that running without background GC cuts their latency to a half or even a third. Multi-hour stress tests that max out all node CPU cores suggest that our current credit_flow default can still be bumped safely, reducing the amount of time publishers are throttled and making publisher latency and node's observed message throughput more predictable.
|
does credit_flow affect consume speed? if there are a few consumers and small prefetch_count. |
|
Yes but I wouldn't expect any significant throughput
changes. Since credit flow settings can be configured already, you can give it a try with your workloads and see for yourself.
… On 5 Feb 2017, at 18:31, ShawnLi ***@***.***> wrote:
does credit_flow affect consume speed? if there are a few consumers and small prefetch_count.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub, or mute the thread.
|
If we increase generic value for all processes, we should also increase them for the message store. The ratio isn't particularly scientific.
michaelklishin
requested review from
gerhard,
dumbbell,
dcorbacho and
kjnilsson
gerhard
approved these changes
Feb 6, 2017
There was a problem hiding this comment.
This looks great, I'll re-create https://www.pivotaltracker.com/story/show/137952203 with these changes.
dcorbacho
approved these changes
Feb 6, 2017
kjnilsson
approved these changes
Feb 6, 2017
dumbbell
approved these changes
Feb 7, 2017
michaelklishin
added a commit
to rabbitmq/rabbitmq-common
that referenced
this pull request
Feb 17, 2017
4000 is not meaningfully different from 3000 but is closer to the new IO_BATCH_SIZE value. References rabbitmq/rabbitmq-server#1098.
michaelklishin
added a commit
that referenced
this pull request
Feb 17, 2017
Avoids a validation warning and if we increase initial message store disk bound credit to 4000, all values become proportionally increased (compared to < 3.6.7 defaults).
michaelklishin
added a commit
to rabbitmq/rabbitmq-common
that referenced
this pull request
Feb 17, 2017
4000 is not meaningfully different from 3000 but is closer to the new IO_BATCH_SIZE value. References rabbitmq/rabbitmq-server#1098.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Quite a bit of evidence from different kinds of workloads suggest
that background GC does not contribute meaningfully to reducing
node RAM usage on most of them. It does, however, on each run
produce a massive spike in VM and CPU context switches,
causing latency spikes.
Some users report that running without background GC cuts
their latency to a half or even down to a third.
Multi-hour stress tests that max out all node CPU cores
suggest that our current credit_flow default can still be bumped
safely, reducing the amount of time publishers are throttled
and making publisher latency and node's observed message throughput
more predictable.