Move to the Kafka 0.10 message format (with timestamps) #381
Conversation
This reverts commit d03eae0.
dasch
changed the title
|
@dasch is there any chance of creating a second branch for 0.10 support, with RubyGems releases? I'd really love to use the broker timestamps to improve our monitoring of Kafka, but that requires the 0.10 API. |
|
I probably won't do Rubygems releases, as that would require me to keep the two branches in sync at all times, but I can try to rebase this branch from time to time and you can depend on it from your Gemfile. Would that work? |
|
Hey @dasch, looks like 0.5.x is targetting the Kafka 0.10 API? Is that correct? If so, anyone wanting to use 0.10 API features can start using the beta of that I think? Maybe worth putting in the README if so. |
|
@isaacseymour I've pulled the beta from Rubygems because I wanted to continue development on the master branch and be able to push 0.9 compatible releases from that (as I'm only able to run 0.9 in production myself.) Once I have a production environment running 0.10+ I'll switch the master branch over to be 0.10 compatible and cut a stable branch for 0.9 support. |
|
Update: planning on cutting a stable branch for 0.4 on Monday, then merging this and releasing v0.5.0! |
| @@ -38,11 +38,14 @@ def initialize(name:, partitions:) | |||
|
|
|||
| attr_reader :topics | |||
|
|
|||
| def initialize(topics: []) | |||
| def initialize(topics: [], throttle_time_ms: 0) | |||
There was a problem hiding this comment.
is the throttle_time_ms going to be configurable?
There was a problem hiding this comment.
It's not set by the client, but rather by the broker. From the docs:
Duration in milliseconds for which the request was throttled due to quota violation. (Zero if the request did not violate any quota.)
There was a problem hiding this comment.
I don't have any plans to add support for quotas, but I'm totally open to PRs :)
#376 was merged, effectively ending backwards compatibility with Kafka 0.9, because I thought we were ready to move off Kafka 0.9. This has proven to not be the case, so I'm not comfortable with having the main line of development (master) only be compatible with a version of Kafka I'm not personally running.
Native Kafka 0.10 support (without protocol format re-writing on the broker side) has been postponed.