Sync rabbitmq.config.example with PROJECT_ENV #1366

gerhard · 2017-09-19T08:39:33Z

I want to reconcile the existing discrepancies between rabbitmq.config.example & PROJECT_ENV. I want to review all config properties and make sure that they are all present and accurate in this canonical rabbitmq.config

Let's discuss which changes need to happen where so that rabbitmq.config.example is accurate & in sync with PROJECT_ENV

gerhard · 2017-09-19T10:07:00Z

Once the missing properties are added to rabbitmq.config.example, I was thinking that we should order all config properties alphabetically. It requires the least cognitive overhead, both when adding or looking properties up. What do y'all think?

rabbitmq/rabbitmq-server#1366

michaelklishin · 2017-09-20T01:38:35Z

docs/rabbitmq.config.example

@@ -338,6 +372,26 @@
   %%
   %% {queue_index_embed_msgs_below, 4096},

+   %% The credits that a queue process is given by the message store.


These paragraphs belong to a guide on rabbitmq.com. Comments to configuration keys can be brief descriptions with a link to more detailed docs.

Here is a good candidate. Might need some rearranging and editing but it tries to cover the same set of settings.

I believe that contextual information is highly valuable when dealing with configuration properties. I agree that guides are essential for explaining how multiple configuration properties work together, but I strongly believe that we should keep essential details alongside configuration properties, in the config file context, without requiring users to jump out.

When I look at a canonical config example, I'm thinking:

What configs exist?

What do they mean?

Why would I want to change a config?

The Cassandra config file example captures my sentiment well, as does this less-known nginx config.

I would really like to know what others think re the above.

Given the big picture, I will reduce the scope of this PR and just sync PROJECT_ENV with rabbitmq.config.example. Single lines are in order, especially since I am yet to understand what every configuration property does.

@gerhard Cassandra's config does not explain what Hinted Handoff is or why it exists in several paragraphs, it says "See http://wiki.apache.org/cassandra/HintedHandoff". A few sentences per key makes sense but config key descriptions should not end up being small guides. There are 2 config examples in master, would you duplicate all those "micro guides"? That's my point.

There are number of guides already and most of config variables are documented in those guides, so we can keep doing that and add links here.

hairyhum · 2017-09-20T13:27:09Z

docs/rabbitmq.config.example

+   %% and then 800 for every 800 messages that it processes.
+   %%
+   %% A queue process persists AMQP messages by sending them 1 by 1 to the message store process.
+   %% If the queue process sent 4000 AMQP messages and the message store process didn't manage to persist 800,


It does not have to persist messages, just process them. Credit flow ack is sent before message store handles a message, it will take some time to actually persist messages.

@michaelklishin

…nfig Having discussed with @michaelklishin & @hairyhum, it was agreed that these details belong in http://www.rabbitmq.com/persistence-conf.html

@michaelklishin

Add links instead, as per discussion with @michaelklishin & @hairyhum

gerhard · 2017-09-20T17:17:17Z

This is now ready to review & merge

This was the last pass, there are no more left in PROJECT_ENV, promise.

michaelklishin · 2017-09-21T15:29:31Z

docs/rabbitmq.config.example

+   %% {mirroring_sync_batch_size, 4096},
+
+   %% Controls flow control between queue mirrors
+   %% Be careful with this configuration, it's controversial


It is not controversial and disabling it is dangerous (as is not having any flow control) with well known outcomes: masters outpacing mirrors at some point, making them consume more and more RAM.

michaelklishin · 2017-09-21T15:30:48Z

docs/rabbitmq.config.example

+   %% {hipe_compile, false},
+
+   %% Number of delegate processes to use for intra-cluster communication.
+   %% On a machine which has a very large number of cores and is also part of a cluster,


How large is "very large"? I'd say "more than 16 cores and plenty of bandwidth".

michaelklishin · 2017-09-21T15:32:03Z

docs/rabbitmq.config.example

+   %% Number of credits that a connection, channel or queue are given
+   %% By default, every connection, channel or queue is given 400 credits,
+   %% and then 200 for every 200 messages that it processes
+   %% See https://www.rabbitmq.com/blog/2015/10/06/new-credit-flow-settings-on-rabbitmq-3-5-5/


We should mention that increasing these values may help with throughput but is dangerous (very high values are no different from having any flow control).

michaelklishin · 2017-09-21T15:33:28Z

docs/rabbitmq.config.example

-   %%                       {nodelay,   true}]}
+   %% {tcp_listen_options, [
+   %%                         {backlog,   128},
+   %%                         {nodelay,       true},


nodelay is forced by Ranch even if you disable it. I see no reason to do that, by the way.

Should nodelay be removed from PROJECT_ENV?

michaelklishin · 2017-09-21T15:34:03Z

docs/rabbitmq.config.example

@@ -572,8 +667,12 @@

   %% TCP/Socket options (as per the broker configuration).
   %%
-   %% {tcp_listen_options, [{backlog,   128},
-   %%                       {nodelay,   true}]}
+   %% {tcp_listen_options, [


Any reason why there is no link to http://www.rabbitmq.com/networking.html here? It's a pretty extensive guide.

michaelklishin · 2017-09-21T15:38:34Z

docs/rabbitmq.config.example

-   %% {background_gc_target_interval, 60000}
+   %% {background_gc_target_interval, 60000},
+
+   %% File size in bytes used for persisting messages


Please change it to "Message store operations are stored in a sequence of files called segments. This controls max size of a segment file. Increasing this value may speed up (sequential) disk writes but will slow down segment GC process. DO NOT CHANGE THIS for existing installations."

michaelklishin · 2017-09-21T15:39:05Z

docs/rabbitmq.config.example

+   %%
+   %% {disk_monitor_failure_retries, 10},
+
+   %% Milliseconds to wait for disk space usage values


This is an interval of time disk monitor waits before retrying, not a query timeout.

michaelklishin · 2017-09-21T15:40:36Z

docs/rabbitmq.config.example


   %% SSL handshake timeout, in milliseconds.
   %%
   %% {ssl_handshake_timeout, 5000},

+   %% Whether to allow SSL POODLE attack,


This makes it sound like anyone wants to enable an attack.

"Makes RabbitMQ accept SSLv3 client connections by default. DO NOT DO THIS IF YOU CAN HELP IT."

gerhard · 2017-09-22T12:08:35Z

@michaelklishin how about now?

Add all log categories to example config mentioned on the configure page

31694c5

gerhard changed the title ~~Add all log categories to example config mentioned on the configure page~~ WIP: Add all log categories to example config mentioned on the configure page Sep 19, 2017

gerhard requested review from dumbbell, michaelklishin and hairyhum September 19, 2017 08:39

gerhard changed the title ~~WIP: Add all log categories to example config mentioned on the configure page~~ WIP: Sync rabbitmq.config.example with https://www.rabbitmq.com/configure.html Sep 19, 2017

gerhard force-pushed the sync-rabbitmq-config-example-with-configure-page branch from 904a177 to a8013ae Compare September 19, 2017 10:01

gerhard force-pushed the sync-rabbitmq-config-example-with-configure-page branch 2 times, most recently from a1a003e to bdb108d Compare September 19, 2017 10:16

gerhard added a commit to rabbitmq/rabbitmq-website that referenced this pull request Sep 19, 2017

Add mirroring_sync_batch_size to RabbitMQ Configuration page

093bddf

rabbitmq/rabbitmq-server#1366

gerhard mentioned this pull request Sep 19, 2017

Add msg_store_credit_disc_bound & msg_store_credit_disc_bound to https://www.rabbitmq.com/configure.html rabbitmq/rabbitmq-website#434

Merged

gerhard force-pushed the sync-rabbitmq-config-example-with-configure-page branch from b5b67ff to bea863b Compare September 19, 2017 15:47

michaelklishin requested changes Sep 20, 2017

View reviewed changes

gerhard changed the title ~~WIP: Sync rabbitmq.config.example with https://www.rabbitmq.com/configure.html~~ WIP: Sync rabbitmq.config.example with PROJECT_ENV Sep 20, 2017

hairyhum reviewed Sep 20, 2017

View reviewed changes

gerhard added 8 commits September 20, 2017 17:29

Add queue_master_locator to example config

9ab7027

Add delegate_count to example config

131843b

Add mirroring_sync_batch_size to example config

1bf1f35

Add msg_store_credit_disc_bound to example config

605eb85

Clarify IO_BATCH_SIZE in betas -> deltas (not gammas!) comment

07b85f6

Explain the IoBatchSize role in push_betas_to_deltas

2f34158

Remove extra details from msg_store_credit_disc_bound from example co…

f54d02d

…nfig Having discussed with @michaelklishin & @hairyhum, it was agreed that these details belong in http://www.rabbitmq.com/persistence-conf.html

Add msg_store_io_batch_size to example config

33d2b98

gerhard force-pushed the sync-rabbitmq-config-example-with-configure-page branch from bea863b to 33d2b98 Compare September 20, 2017 16:29

gerhard added 2 commits September 20, 2017 17:35

Remove extra details from new config properties added to example config

45f8143

Add links instead, as per discussion with @michaelklishin & @hairyhum

Add other missing config properties from PROJECT_ENV to example config

78d2e89

gerhard changed the title ~~WIP: Sync rabbitmq.config.example with PROJECT_ENV~~ Sync rabbitmq.config.example with PROJECT_ENV Sep 20, 2017

Add a few extra missing properties & correct defaults in example config

e4b2fe7

gerhard added 4 commits September 20, 2017 21:37

Add last missing config properties to example config

a521119

This was the last pass, there are no more left in PROJECT_ENV, promise.

Fix typo: SSH -> SSFix typo: SSH -> SSL

19c4dd9

Remove redundant for

83e6c23

Replace commented line with empty line

b68a37b

michaelklishin requested changes Sep 21, 2017

View reviewed changes

Additions & changes to example config as per rabbitmq-server#1366

4398ab1

Edits, more links

87b51a6

michaelklishin approved these changes Sep 22, 2017

View reviewed changes

michaelklishin merged commit 879d183 into stable Sep 22, 2017

gerhard deleted the sync-rabbitmq-config-example-with-configure-page branch October 2, 2017 09:05

Sync rabbitmq.config.example with PROJECT_ENV #1366

Sync rabbitmq.config.example with PROJECT_ENV #1366

Uh oh!

Conversation

gerhard commented Sep 19, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gerhard commented Sep 19, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

michaelklishin Sep 20, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gerhard commented Sep 20, 2017

Uh oh!

michaelklishin Sep 21, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gerhard commented Sep 22, 2017

Uh oh!

Uh oh!

gerhard commented Sep 19, 2017 •

edited

Loading

gerhard commented Sep 19, 2017 •

edited

Loading

michaelklishin Sep 20, 2017 •

edited

Loading

michaelklishin Sep 21, 2017 •

edited

Loading