Browse code

kafka: README file

Vicente Hernando authored on 06/11/2019 20:23:52
Showing 1 changed files
1 1
new file mode 100644
... ...
@@ -0,0 +1,233 @@
0
+Kafka Module
1
+
2
+Vicente Hernando
3
+
4
+   <vhernando@sonoc.io>
5
+
6
+Edited by
7
+
8
+Vicente Hernando
9
+
10
+   <vhernando@sonoc.io>
11
+
12
+Javier Gallart
13
+
14
+   <jgallart@sonoc.io>
15
+
16
+   Copyright © 2019 www.sonoc.io
17
+     __________________________________________________________________
18
+
19
+   Table of Contents
20
+
21
+   1. Admin Guide
22
+
23
+        1. Overview
24
+        2. Dependencies
25
+
26
+              2.1. Kamailio Modules
27
+              2.2. External Libraries or Applications
28
+              2.3. Parameters
29
+
30
+                    2.3.1. brokers (string)
31
+                    2.3.2. configuration (string)
32
+                    2.3.3. topic (string)
33
+
34
+              2.4. Functions
35
+
36
+                    2.4.1. kafka_send(topic, msg)
37
+
38
+              2.5. RPC Commands
39
+
40
+                    2.5.1. kafka.stats
41
+                    2.5.2. kafka.stats_topic
42
+
43
+   List of Examples
44
+
45
+   1.1. Set brokers parameter
46
+   1.2. Set configuration parameter
47
+   1.3. Set topic parameter
48
+   1.4. kafka_send usage
49
+   1.5. kafka.stats usage
50
+   1.6. kafka.stats usage
51
+
52
+Chapter 1. Admin Guide
53
+
54
+   Table of Contents
55
+
56
+   1. Overview
57
+   2. Dependencies
58
+
59
+        2.1. Kamailio Modules
60
+        2.2. External Libraries or Applications
61
+        2.3. Parameters
62
+
63
+              2.3.1. brokers (string)
64
+              2.3.2. configuration (string)
65
+              2.3.3. topic (string)
66
+
67
+        2.4. Functions
68
+
69
+              2.4.1. kafka_send(topic, msg)
70
+
71
+        2.5. RPC Commands
72
+
73
+              2.5.1. kafka.stats
74
+              2.5.2. kafka.stats_topic
75
+
76
+1. Overview
77
+
78
+   This module produces and sends messages to a Kafka server.
79
+
80
+2. Dependencies
81
+
82
+   2.1. Kamailio Modules
83
+   2.2. External Libraries or Applications
84
+   2.3. Parameters
85
+
86
+        2.3.1. brokers (string)
87
+        2.3.2. configuration (string)
88
+        2.3.3. topic (string)
89
+
90
+   2.4. Functions
91
+
92
+        2.4.1. kafka_send(topic, msg)
93
+
94
+   2.5. RPC Commands
95
+
96
+        2.5.1. kafka.stats
97
+        2.5.2. kafka.stats_topic
98
+
99
+2.1. Kamailio Modules
100
+
101
+   The following modules must be loaded before this module:
102
+     * none.
103
+
104
+2.2. External Libraries or Applications
105
+
106
+   The following libraries or applications must be installed before
107
+   running Kamailio with this module loaded:
108
+     * librdkafka: the Apache Kafka C/C++ client library.
109
+       https://github.com/edenhill/librdkafka
110
+       Old librdkafka version like 1.1.0 has been reported to work. Newer
111
+       versions should work fine (E.g: Version 1.2.2-RC1 also works OK)
112
+       https://github.com/edenhill/librdkafka/releases
113
+
114
+2.3. Parameters
115
+
116
+2.3.1. brokers (string)
117
+
118
+   Specifies a list of brokers separated by commas.
119
+
120
+   From librdkafka documentation:
121
+
122
+   brokerlist is a ,-separated list of brokers in the format:
123
+   <broker1>,<broker2>,
124
+
125
+   Where each broker is in either the host or URL based format:
126
+     * <host>[:<port>]
127
+     * <proto>://<host>[:port]
128
+
129
+   <proto> is either PLAINTEXT, SSL, SASL, SASL_PLAINTEXT
130
+
131
+   The two formats can be mixed but ultimately the value of the
132
+   security.protocol config property decides what brokers are allowed.
133
+
134
+   This parameter is mandatory. There is no default value.
135
+
136
+   Example 1.1. Set brokers parameter
137
+...
138
+modparam("kafka", "brokers", "localhost:9092")
139
+modparam("kafka", "brokers", "broker1:10000,broker2")
140
+modparam("kafka", "brokers", "SSL://broker3:9000,ssl://broker2")
141
+...
142
+
143
+2.3.2. configuration (string)
144
+
145
+   Specifies a set of general properties.
146
+
147
+   Each configuration property follows: name = value pattern. And
148
+   configuration properties are separated by ;
149
+
150
+   This parameter is optional, but if it exists it can be configured only
151
+   once.
152
+
153
+   Example 1.2. Set configuration parameter
154
+...
155
+modparam("kafka", "configuration", "topic.metadata.refresh.interval.ms=20000;que
156
+ue.buffering.max.messages=1000000;metadata.request.timeout.ms=90000")
157
+
158
+modparam("kafka", "configuration", "topic.metadata.refresh.interval.ms=20000;que
159
+ue.buffering.max.messages=500000;debug=all;metadata.request.timeout.ms=900000")
160
+...
161
+
162
+2.3.3. topic (string)
163
+
164
+   Specifies a topic name and a set of topic properties.
165
+
166
+   The topic defined in topic parameter has to already exist in Kafka
167
+   servers.
168
+
169
+   Each topic property is a list of attribute = value separated by
170
+   semicolon.
171
+
172
+   name atribute indicates the topic name. It is mandatory. Other
173
+   attributes mean names of properties and are optional.
174
+
175
+   This parameter is optional. Each topic needs a topic parameter so
176
+   several topic parameters are allowed.
177
+
178
+   Example 1.3. Set topic parameter
179
+...
180
+modparam("kafka", "topic", "name=my_topic;request.required.acks=0;request.timeou
181
+t.ms=10000")
182
+modparam("kafka", "topic", "name=second_topic;request.required.acks=0;request.ti
183
+meout.ms=10000")
184
+modparam("kafka", "topic", "name=third_topic")
185
+...
186
+
187
+2.4. Functions
188
+
189
+2.4.1.  kafka_send(topic, msg)
190
+
191
+   Send a message to a specific topic via Kafka server.
192
+
193
+   This function returns -1 for all sort of errors. (So execution of
194
+   script continues)
195
+
196
+   Parameters:
197
+     * topic: (string) name of the topic. It is mandatory.
198
+     * msg: (string) message to send. It is mandatory.
199
+
200
+   Available via KEMI framework as kafka.send.
201
+
202
+   Example 1.4. kafka_send usage
203
+...
204
+# Send "test message" to topic "my_topic"
205
+kafka_send("my_topic", "test message");
206
+...
207
+
208
+2.5. RPC Commands
209
+
210
+2.5.1. kafka.stats
211
+
212
+   Show statistics about total sent messages and failed to deliver ones.
213
+
214
+   Example 1.5. kafka.stats usage
215
+...
216
+kamcmd kafka.stats
217
+Total messages: 26  Errors: 0
218
+...
219
+
220
+2.5.2. kafka.stats_topic
221
+
222
+   Show statistics about sent messages and failed to deliver ones for a
223
+   specific topic.
224
+
225
+   Parameter: topic (string) name of the topic. Required.
226
+
227
+   Example 1.6. kafka.stats usage
228
+...
229
+# Show statistics for my_topic.
230
+kamcmd kafka.stats_topic "my_topic"
231
+Topic: my_topic  Total messages: 17  Errors: 0
232
+...