Mapping bindings between a stream and a pub-sub exchange causes publishers to fails and channels on publishers to explode in count

[jefftrimm]

Mapping bindings between a stream and a pub-sub exchange causes publishers to fails and channels on publishers to explode in count.

Step to reproduce:

1. 2-node cluster.
2. Create a pub-sub exchange.
3. Create a regular mirrored queue.
4. Create one or more topic bindings to the queue.
5. Have a publisher with a persistent connection to RabbitMQ publish events
6. Events will be delivered to classic queue with no issues
7. Create a stream queue, in my case I set a max-age of events to 1D, though it's unclear if that matters.
8. Bind the stream queue to the pub-sub exchange with the same topic bindings as the classic queue.
9. Attempt to publish more messages from the publisher which is still connected to the broker.
10. Events will show up in the stream, but the publisher client never seems to successfully publish and the number of channels in the connection will continue to grow until the max channel limit of 2047 channels is reached.

[michaelklishin]

I will convert this issue to a GitHub discussion. Currently GitHub will automatically close and lock the issue even though your question will be transferred and responded to elsewhere. This is to let you know that we do not intend to ignore this but this is how the current GitHub conversion mechanism makes it seem for the users :(

[michaelklishin]

We need server logs and an executable way to reproduce. RabbitMQ core never opens channels (some plugins do), so the channel build-up may be a behavior caused by your publisher code. It's also not clear how can said events "show up in the stream" while at the same time "publishers never seem to successfully publish". What is the success criteria used?

An executable way to reproduce and logs would answer these questions and more.

[acogoluegnes]

I ran a test under the following conditions:

• RabbitMQ 3.9.12
• 3-node cluster
• a stream and a mirrored queue bound to a topic exchange with the same routing key
• a Java program using the AMQP Java client 5.14.0 that publishes to the given exchange, with confirm.select

The published messages are confirmed and I did not observe any channel leakage. The messages end up in both the queue and the stream. This is the expected behavior.

Could you provide an initialization procedure and the publisher code to reproduce?

[jefftrimm]

Is 3 nodes a must-have for correct operation with streams in this use case? I was testing this on an existing 2-node cluster in our QA environment. I know 3 nodes are needed for proper Quorum and HA, but I was testing this on a 2 node cluster for functional POC.

[acogoluegnes]

Yes, a 3-node cluster is strongly recommended. Quorum queues and streams uses Raft internally (QQ for their content, streams for topology and leader election, not for data itself), and odd number of nodes (3, 5, 7) are better for these. The MQTT plugin also uses Raft at some point.