You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: src/docs/asciidoc/api.adoc
+46Lines changed: 46 additions & 0 deletions
Original file line number
Diff line number
Diff line change
@@ -851,6 +851,10 @@ Useful when using an external store for offset tracking.
851
851
|Number of credits when the subscription is created.
852
852
Increase for higher throughput at the expense of memory usage.
853
853
|1
854
+
855
+
|`flow#strategy`
856
+
|The `ConsumerFlowStrategy` to use.
857
+
|`ConsumerFlowStrategy#creditOnChunkArrival(1)`
854
858
|===
855
859
856
860
[NOTE]
@@ -1099,6 +1103,48 @@ When a glitch happens and triggers the re-subscription, the server-side stored o
1099
1103
Using this server-side stored offset can lead to duplicates, whereas using the in-memory, application-specific offset tracking variable is more accurate.
1100
1104
A custom `SubscriptionListener` lets the application developer uses what's best for the application if the computed value is not optimal.
1101
1105
1106
+
===== Flow Control
1107
+
1108
+
This section covers how a consumer can tell the broker when to send more messages.
1109
+
1110
+
By default, the broker keeps sending messages as long as messages are processed and the `MessageHandler#handle(Context, Message)` method returns.
1111
+
This strategy works fine if message processing is fast enough.
1112
+
If message processing takes longer, one can be tempted to process messages in parallel with an `ExecutorService`.
1113
+
This will make the `handle` method return immediately and the broker will keep sending messages, potentially overflowing the consumer.
1114
+
1115
+
What we miss in the parallel processing case is a way to tell the library we are done processing a message and that we are ready at some point to handle more messages.
1116
+
This is the goal of the `MessageHandler.Context#processed()` method.
1117
+
1118
+
This method is by default a no-op because the default flow control strategy keeps asking for more messages as soon as message processing is done.
1119
+
This method gets some real behavior to control the flow of messages when an appropriate `ConsumerFlowStrategy` is set `ConsumerBuilder#flow()`.
1120
+
The following code snippet shows how to set a handy consumer flow strategy:
In the example we set up the `creditWhenHalfMessagesProcessed` strategy which asks for more messages once half of the current messages have been marked as processed.
1131
+
The broker does not send messages one by one, it sends <<chunk-definition,chunks>> of messages.
1132
+
A chunk of messages can contain 1 to several thousands of messages.
1133
+
So with the strategy set above, once `processed()` has been called for half of the messages of the current chunk, the library will ask the broker for another one (it will provide a _credit_ for the subscription).
1134
+
By doing this, the next chunk should arrive by the time we are done with the other half of the current chunk.
1135
+
This way the consumer is neither overwhelmed nor idle.
1136
+
1137
+
The `ConsumerFlowStrategy` interface provides some static helpers to configure the appropriate strategy.
1138
+
1139
+
Additional notes on consumer flow control:
1140
+
1141
+
* Make sure to **call the `processed()` method** once you set up a `ConsumerFlowStrategy`.
1142
+
The method is a no-op by default, but it is essential to call it with count-based strategies like `creditWhenHalfMessagesProcessed` or `creditOnProcessedMessageCount`.
1143
+
No calling it will stop the dispatching of messages.
1144
+
* Make sure to call `processed()` only once.
1145
+
Whether the method is idempotent depends on the flow strategy implementation.
1146
+
Apart from the default one, the implementations the library provides does not make `processed()` idempotent.
0 commit comments