Skip to content

Default to classic queues v2 #8308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 25, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions deps/rabbit/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,8 @@ _APP_ENV = """[
]},
{halt_on_upgrade_failure, true},
{ssl_apps, [asn1, crypto, public_key, ssl]},
%% classic queue storage implementation version
{classic_queue_default_version, 2},
%% see rabbitmq-server#114
{mirroring_flow_control, true},
{mirroring_sync_batch_size, 4096},
Expand Down
2 changes: 2 additions & 0 deletions deps/rabbit/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ define PROJECT_ENV
]},
{halt_on_upgrade_failure, true},
{ssl_apps, [asn1, crypto, public_key, ssl]},
%% classic queue storage implementation version
{classic_queue_default_version, 2},
%% see rabbitmq-server#114
{mirroring_flow_control, true},
{mirroring_sync_batch_size, 4096},
Expand Down
2 changes: 1 addition & 1 deletion deps/rabbit/priv/schema/rabbit.schema
Original file line number Diff line number Diff line change
Expand Up @@ -2472,7 +2472,7 @@ end}.

{translation, "rabbit.classic_queue_default_version",
fun(Conf) ->
case cuttlefish:conf_get("classic_queue.default_version", Conf, 1) of
case cuttlefish:conf_get("classic_queue.default_version", Conf, 2) of
1 -> 1;
2 -> 2;
_ -> cuttlefish:unset()
Expand Down
4 changes: 2 additions & 2 deletions deps/rabbit/src/rabbit_amqqueue_process.erl
Original file line number Diff line number Diff line change
Expand Up @@ -470,12 +470,12 @@ init_queue_mode(Mode, State = #q {backing_queue = BQ,

init_queue_version(Version0, State = #q {backing_queue = BQ,
backing_queue_state = BQS}) ->
%% When the version is undefined we use the default version 1.
%% When the version is undefined we use the default version 2.
%% We want to BQ:set_queue_version in all cases because a v2
%% policy might have been deleted, for example, and we want
%% the queue to go back to v1.
Version = case Version0 of
undefined -> rabbit_misc:get_env(rabbit, classic_queue_default_version, 1);
undefined -> rabbit_misc:get_env(rabbit, classic_queue_default_version, 2);
_ -> Version0
end,
BQS1 = BQ:set_queue_version(Version, BQS),
Expand Down
3 changes: 2 additions & 1 deletion deps/rabbit/src/rabbit_variable_queue.erl
Original file line number Diff line number Diff line change
Expand Up @@ -490,10 +490,11 @@ process_recovery_terms(Terms) ->
PRef -> {PRef, Terms}
end.

%% If queue-version is undefined, we assume v2 starting with RabbitMQ 3.13.0.
queue_version(Q) ->
Resolve = fun(_, ArgVal) -> ArgVal end,
case rabbit_queue_type_util:args_policy_lookup(<<"queue-version">>, Resolve, Q) of
undefined -> rabbit_misc:get_env(rabbit, classic_queue_default_version, 1);
undefined -> rabbit_misc:get_env(rabbit, classic_queue_default_version, 2);
Vsn when is_integer(Vsn) -> Vsn;
Vsn -> binary_to_integer(Vsn)
end.
Expand Down
6 changes: 3 additions & 3 deletions deps/rabbit/test/dynamic_ha_SUITE.erl
Original file line number Diff line number Diff line change
Expand Up @@ -1018,16 +1018,16 @@ apply_policy(Config, N, undefined) ->
apply_policy(Config, N, all) ->
rabbit_ct_broker_helpers:set_ha_policy(
Config, N, ?POLICY, <<"all">>,
[{<<"ha-sync-mode">>, <<"automatic">>}]);
[{<<"ha-sync-mode">>, <<"automatic">>}, {<<"queue-mode">>, <<"lazy">>}]);
apply_policy(Config, N, {nodes, Nodes}) ->
NNodes = [atom_to_binary(Node) || Node <- Nodes],
rabbit_ct_broker_helpers:set_ha_policy(
Config, N, ?POLICY, {<<"nodes">>, NNodes},
[{<<"ha-sync-mode">>, <<"automatic">>}]);
[{<<"ha-sync-mode">>, <<"automatic">>}, {<<"queue-mode">>, <<"lazy">>}]);
apply_policy(Config, N, {exactly, Exactly}) ->
rabbit_ct_broker_helpers:set_ha_policy(
Config, N, ?POLICY, {<<"exactly">>, Exactly},
[{<<"ha-sync-mode">>, <<"automatic">>}]).
[{<<"ha-sync-mode">>, <<"automatic">>}, {<<"queue-mode">>, <<"lazy">>}]).

forget_cluster_node(Config, Node, NodeToRemove) ->
rabbit_ct_broker_helpers:rabbitmqctl(
Expand Down
156 changes: 156 additions & 0 deletions release-notes/3.13.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
## RabbitMQ 3.13.0

RabbitMQ `3.12.0` is a new feature release.

## Highlights

This release includes several new features, optimizations, and graduates (makes mandatory) a number of feature flags.

The user-facing areas that have seen the biggest improvements in this release are

* Classic queues use version 2 of the format format. This should significantly improve performance.

This release also features many internal API improvements in preparation to 4.0
with [Khepri](https://www.youtube.com/watch?v=huT-zmXvfuM).

See Compatibility Notes below to learn about breaking or potentially breaking changes in this release.

## Release Artifacts

RabbitMQ releases are distributed via [GitHub](https://github.com/rabbitmq/rabbitmq-server/releases).
[Debian](https://rabbitmq.com/install-debian.html) and [RPM packages](https://rabbitmq.com/install-rpm.html) are available via Cloudsmith mirrors, as well as [PackageCloud](https://packagecloud.io/rabbitmq).

[Community Docker image](https://hub.docker.com/_/rabbitmq/), [Chocolatey package](https://community.chocolatey.org/packages/rabbitmq), and the [Homebrew formula](https://rabbitmq.com/install-homebrew.html)
are other installation options. They are updated with a delay (usually a few days).


## Erlang/OTP Compatibility Notes

This release [requires Erlang 25.0](https://www.rabbitmq.com/which-erlang.html) or later.
This introduces feature parity for x86- and ARM64-based CPUs: Erlang 25 offers the JIT and
[modern Flame Graph profiling](https://blog.rabbitmq.com/posts/2022/05/flame-graphs/) tooling
for both of those major CPU architectures.

[Provisioning Latest Erlang Releases](https://www.rabbitmq.com/which-erlang.html#erlang-repositories) explains
what package repositories and tools can be used to provision latest patch versions of Erlang 25.x.

## Upgrading to 3.13

### Documentation guides on upgrades

See the [Upgrading guide](https://www.rabbitmq.com/upgrade.html) for documentation on upgrades and [RabbitMQ change log](https://www.rabbitmq.com/changelog.html)
for release notes of other releases.

### Required Feature Flags


### Mixed version cluster compatibility

RabbitMQ 3.13.0 nodes can run alongside `3.12.x` nodes. `3.12.x`-specific features can only be made available when all nodes in the cluster
upgrade to 3.13.0 or any other patch release in the new series.

While operating in mixed version mode, some aspects of the system may not behave as expected. The list of known behavior changes is covered below.
Once all nodes are upgraded to 3.13.0, these irregularities will go away.

Mixed version clusters are a mechanism that allows rolling upgrade and are not meant to be run for extended
periods of time (no more than a few hours).



## Compatibility Notes

### More Feature Flags Gratuate to Core Features ("Always Enabled")


### Minimum Supported Erlang Version

Starting with this release, RabbitMQ requires Erlang 25.0 or later versions. Nodes **will fail to start**
on older Erlang releases.

Erlang 25 as our new baseline means much improved performance on ARM64 architectures, [profiling with flame graphs](https://blog.rabbitmq.com/posts/2022/05/flame-graphs/)
across all architectures, and the most recent TLS 1.3 implementation available to all RabbitMQ 3.11 users.


### Client Library Compatibility

Client libraries that were compatible with RabbitMQ `3.12.x` will be compatible with `3.13.0`.


### Getting Help

Any questions about this release, upgrades or RabbitMQ in general are welcome on the [RabbitMQ mailing list](https://groups.google.com/forum/#!forum/rabbitmq-users).


## Changes Worth Mentioning

Release notes are kept under [rabbitmq-server/release-notes](https://github.com/rabbitmq/rabbitmq-server/tree/v3.11.x/release-notes).

### Core Server

#### Enhancements

* Reduced memory footprint, improved memory use predictability and throughput of classic queues (version 2, or CQv2).
This particularly benefits classic queues with longer backlogs.

Classic queue v2 (CQv2) storage implementation **is now the default**. It is possible to switch
the default back to CQv1 using `rabbitmq.conf`:

``` ini
# uses CQv1 by default
classic_queue.default_version = 1
```

Individual queues can be declared by passing `x-queue-version` argument and/or through a `queue-version` policy.

#### Bug Fixes

This release includes all bug fixes shipped in the `3.12.x` series.



### CLI Tools

#### Enhancements


#### Bug Fixes



### MQTT Plugin

#### Enhancements


### Management Plugin

#### Enhancements


#### Bug Fixes


### OAuth 2 AuthN/AuthZ Backend Plugin

#### Enhancement


### HTTPS AuthN/AuthZ Backend Plugin

#### Bug Fixes


### Consul Peer Discovery Plugin

#### Bug Fixes



### Dependency Changes



## Source Code Archives

To obtain source code of the entire distribution, please download the archive named `rabbitmq-server-3.13.0.tar.xz`
instead of the source tarball produced by GitHub.