Shutdown cannot fail #1396
Shutdown cannot fail #1396
Conversation
|
79fa87f doesn't explain the intent but I bet it is: we don't want node shutdown to fail with obscure implementation-specific errors. We want the node to shut down successfully so that tools such as Monit or systemd do not freak out for no good reason. So if you consider that, this is an improvement. |
|
A
In all the above scenarios, the thing that manages RabbitMQ fails because it believes that RabbitMQ is stopped (i.e. shutdown succeeded) while in actual fact the How should we communicate to the thing that manages RabbitMQ that the Erlang VM failed to stop? |
|
I think our best bet is a (possibly non-standard?) process exit code.
… On 18 Oct 2017, at 15:17, Gerhard Lazu ***@***.***> wrote:
A rabbitmqctl shutdown can fail due to the following reasons:
Erlang cookie does not match
RabbitMQ is booting
RabbitMQ is stopping
Erlang VM is crashing (in some cases, it was observed that this can take many hours)
The Erlang VM is running but the distribution is not working correctly - this happens more often than you think in hostile environments, like almost every enterprise, multi-tenant RabbitMQ cluster.
In all the above scenarios, the thing that manages RabbitMQ fails because it believes that RabbitMQ is stopped (i.e. shutdown succeeded) while in actual fact the beam process is still running.
How should we communicate to the thing that manages RabbitMQ that the Erlang VM failed to stop?
—
You are receiving this because you modified the open/close state.
Reply to this email directly, view it on GitHub, or mute the thread.
|
|
This means that A few points to remember:
|
This way tools that manage RabbitMQ nodes can detect this condition among a bunch of fairly generic exit codes. Note that when a timeout occurs, we still use a "temporary failure" code. See #1396 for context.
This is a cherry-pick of commit 79fa87f from
master. Removes a test that commit 904744d appears to have broken.