|
1 | 1 | # hw-kafka-client
|
| 2 | + |
2 | 3 | [](https://circleci.com/gh/haskell-works/hw-kafka-client)
|
3 | 4 |
|
4 | 5 | Kafka bindings for Haskell backed by the
|
5 | 6 | [librdkafka C module](https://github.com/edenhill/librdkafka).
|
6 | 7 |
|
7 |
| -## Credits |
8 |
| -This project is inspired by [Haskakafka](https://github.com/cosbynator/haskakafka) |
9 |
| -which unfortunately doesn't seem to be actively maintained. |
10 |
| - |
11 | 8 | ## Ecosystem
|
12 |
| -HaskellWorks Kafka ecosystem is described here: https://github.com/haskell-works/hw-kafka |
13 | 9 |
|
14 |
| -# Consumer |
| 10 | +HaskellWorks Kafka ecosystem is described here: <https://github.com/haskell-works/hw-kafka> |
| 11 | + |
| 12 | +## Consumer |
| 13 | + |
15 | 14 | High level consumers are supported by `librdkafka` starting from version 0.9.
|
16 | 15 | High-level consumers provide an abstraction for consuming messages from multiple
|
17 | 16 | partitions and topics. They are also address scalability (up to a number of partitions)
|
18 | 17 | by providing automatic rebalancing functionality. When a new consumer joins a consumer
|
19 | 18 | group the set of consumers attempt to "rebalance" the load to assign partitions to each consumer.
|
20 | 19 |
|
21 |
| -### Example: |
| 20 | +### Consumer example |
| 21 | + |
| 22 | +See [Running integration tests locally](#running-integration-tests-locally) to learn how to configure a local environment. |
22 | 23 |
|
23 | 24 | ```bash
|
24 |
| -$ stack build --flag hw-kafka-client:examples |
| 25 | +cabal build --flag examples |
25 | 26 | ```
|
26 | 27 |
|
27 | 28 | or
|
28 | 29 |
|
29 | 30 | ```bash
|
30 |
| -$ stack build --exec kafka-client-example --flag hw-kafka-client:examples |
| 31 | +cabal run kafka-client-example --flag examples |
31 | 32 | ```
|
32 | 33 |
|
33 | 34 | A working consumer example can be found here: [ConsumerExample.hs](example/ConsumerExample.hs)</br>
|
@@ -75,7 +76,7 @@ processMessages kafka = do
|
75 | 76 | return $ Right ()
|
76 | 77 | ```
|
77 | 78 |
|
78 |
| -# Producer |
| 79 | +## Producer |
79 | 80 |
|
80 | 81 | `kafka-client` producer supports sending messages to multiple topics.
|
81 | 82 | Target topic name is a part of each message that is to be sent by `produceMessage`.
|
@@ -117,9 +118,9 @@ producerProps = brokersList [BrokerAddress "localhost:9092"]
|
117 | 118 | In the example above when the producer cannot deliver the message to Kafka,
|
118 | 119 | the error will be printed (and the message will be dropped).
|
119 | 120 |
|
120 |
| -### Example |
| 121 | +### Producer example |
121 | 122 |
|
122 |
| -```Haskell |
| 123 | +```haskell |
123 | 124 | {-# LANGUAGE OverloadedStrings #-}
|
124 | 125 | import Control.Exception (bracket)
|
125 | 126 | import Control.Monad (forM_)
|
@@ -166,6 +167,7 @@ mkMessage k v = ProducerRecord
|
166 | 167 | ```
|
167 | 168 |
|
168 | 169 | ### Synchronous sending of messages
|
| 170 | + |
169 | 171 | Because of the asynchronous nature of librdkafka, there is no API to provide
|
170 | 172 | synchronous production of messages. It is, however, possible to combine the
|
171 | 173 | delivery reports feature with that of callbacks. This can be done using the
|
@@ -234,3 +236,8 @@ After that, tests can be run as usual:
|
234 | 236 | ```
|
235 | 237 | $ cabal test --test-show-details=direct
|
236 | 238 | ```
|
| 239 | + |
| 240 | +## Credits |
| 241 | + |
| 242 | +This project is inspired by [Haskakafka](https://github.com/cosbynator/haskakafka) |
| 243 | +which unfortunately doesn't seem to be actively maintained. |
0 commit comments