Kafka consumer not reading any message with zero timeout_ms poll value

[berrfred]

Zero timeout_ms poll value is working fine with 2.0.2 and 2.0.6 but not any longer with latest releases (2.1.x, 2.2.x).

Using a non zero value solves the issue but in my very specific environment (HPE NonStop) using a non zero timeout_ms poll value leads to a quite higher cpu consumption, experienced years ago with 2.0.2 and also checked after running for a while with 2.2.3 .

[berrfred]

@dpkp Thank you so much for your support, unfortunately it does not seem to solve the 0 timeout_ms issue since I put a record in the queue at 08:35 but it was never read until I changed the timeout_ms from 0 to 100 at 08:38.

05/05/2025; 08:37:00.97; [I] $GMT22 ; no record available
05/05/2025; 08:37:06.13; [I] $GMT22 ; no record available
05/05/2025; 08:37:11.29; [I] $GMT22 ; no record available
05/05/2025; 08:37:16.37; [I] $GMT22 ; no record available
05/05/2025; 08:37:21.42; [I] $GMT22 ; no record available
05/05/2025; 08:37:26.46; [I] $GMT22 ; no record available
05/05/2025; 08:37:31.61; [I] $GMT22 ; no record available
05/05/2025; 08:37:36.65; [I] $GMT22 ; no record available
05/05/2025; 08:37:41.69; [I] $GMT22 ; no record available
05/05/2025; 08:37:46.73; [I] $GMT22 ; no record available
05/05/2025; 08:37:51.77; [I] $GMT22 ; no record available
05/05/2025; 08:37:56.82; [I] $GMT22 ; no record available

05/05/2025; 08:38:00.01; [W] $GMT22 ; statistics - saved
05/05/2025; 08:38:00.04; [W] $GMT22 ; parameters - reloaded

05/05/2025; 08:38:01.92; [I] $GMT22 ; no record available
05/05/2025; 08:38:07.08; [I] $GMT22 ; no record available
05/05/2025; 08:38:12.21; [I] $GMT22 ; no record available
05/05/2025; 08:38:17.32; [I] $GMT22 ; no record available
05/05/2025; 08:38:22.42; [I] $GMT22 ; no record available
05/05/2025; 08:38:27.53; [I] $GMT22 ; no record available
05/05/2025; 08:38:32.69; [W] $GMT22 ; read records - committed (1)
05/05/2025; 08:38:32.69; [W] $GMT22 ; read|oamds-mtonline|1|189231|b'no_check_1000200ffff33gfff'|10000
05/05/2025; 08:38:36.70; [I] $GMT22 ; nowait response for tag 10000
05/05/2025; 08:38:36.84; [I] $GMT22 ; no record available
05/05/2025; 08:38:41.94; [I] $GMT22 ; no record available
05/05/2025; 08:38:47.10; [I] $GMT22 ; no record available
05/05/2025; 08:38:52.24; [W] $GMT22 ; read records - committed (1)
05/05/2025; 08:38:52.24; [W] $GMT22 ; read|oamds-mtonline|1|189232|b'no_check_1000200ffff33gfff'|10001
05/05/2025; 08:38:56.25; [I] $GMT22 ; nowait response for tag 10001
05/05/2025; 08:38:56.38; [I] $GMT22 ; no record available

[berrfred]

Release 2.2.6 - The issue is still present but it keeps working fine with 100ms timeout (drawback of unnecessary cpu consumption in my environment).

[berrfred]

Once started the consumers and calling poll() with timeout_ms=0 (and facing the issue of not reading any message), changing the timeout_ms to 100 solves the issue since it starts reading messages on the next calls to poll() ... and apparently it keeps reading also after reverting the parameter back to 0.

Could not grab the debug logging yet but if this can help I'll try to reproduce it with corresponding logs in the next days.

[dpkp]

timeout_ms=0 should work, but you will need to call it many times before all of the connection + group setup is completed and messages start flowing.