Addon: bootstrap.kafka service with current brokers as endpoint #52
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
i.e. to connect to any kafka broker and get a list of actual broker DNS names to talk to.
so this solves the problem with maintaining the BOOTSTRAP string
solsson
force-pushed
the
addon-bootstrap-service
branch
from
583fcaf to
fabb292
Compare
|
The more I mess with bootstrap.servers strings the better I like this idea. For example in production you'd want a couple of brokers, but that causes error messages in scaled down clusters like #44. Let's merge it, as existing bootstrap settings will work as before. |
|
|
Actually I will have to revert the test change in this PR, and merge only the service. I've tested it during #84 but need to merge this to master without conflicts in ongoing test refactoring. |
solsson
added a commit
that referenced
this pull request
Nov 7, 2017
and trust alarms on Under-replicated Partitions to let us know when something is really wrong. Do clients actually care about Readiness? The bootstrap service (#52) will definitely care, which is good. The `broker` service, that the StatefulSet manifest depends on (https://github.com/Yolean/kubernetes-kafka/blob/v2.1.0/50kafka.yml#L7) for naming, is without `publishNotReadyAddresses`. Clients will bootstrap, get the individual DNS names of brokers, resolve those addresses and connect directly to pods.
This reverts commit fabb292.
solsson
added a commit
that referenced
this pull request
Nov 9, 2017
The test should still pass if any single broker is down.
solsson
added a commit
that referenced
this pull request
Dec 18, 2017
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This looks useful but I'm hesitant to merge to master due to #21, which we dealt with by removing the service in #30.
I fail to figure out how to test, let alone enforce, that this service is only used for the initial connection but not for actual consumption or production. Clients should discover brokers through this service, but not suffer from the round-robin nature of it. A danger is that when developing with 1 replica, #44 for example, there is no round-robin and thus no such issues.