Browse code

modules: readme files regenerated - db_cassandra ... [skip ci]

Kamailio Dev authored on 26/09/2019 07:31:48
Showing 1 changed files
... ...
@@ -61,6 +61,13 @@ Chapter 1. Admin Guide
61 61
 
62 62
 1. Overview
63 63
 
64
+   Note: the module requires old version of external library, not
65
+   compiling with those available out of the stock in the Linux
66
+   distributions. It is going to be kept for a while in case someone wants
67
+   to pick it up and upgrade. Also, the module was never extensively
68
+   tested, therefore take the appropriate actions in case you plan to use
69
+   it.
70
+
64 71
    Db_cassandra is one of the Kamailio database modules. It does not
65 72
    export any functions executable from the configuration scripts, but it
66 73
    exports a subset of functions using the database API, and thus, other
Browse code

modules: readme files regenerated - acc ... [skip ci]

Kamailio Dev authored on 28/02/2018 17:03:37 • The Root committed on 28/02/2018 19:11:36
Showing 1 changed files
... ...
@@ -174,7 +174,7 @@ default_validation_class='UTF8Type' and key_validation_class='UTF8Type';
174 174
 
175 175
 5. Installation
176 176
 
177
-   Because the db_cassandra module dependes on an external library, it is
177
+   Because the db_cassandra module depends on an external library, it is
178 178
    not compiled and installed by default. You can use one of these
179 179
    options:
180 180
      * - edit the "Makefile" and remove "db_cassandra" from
Browse code

core, lib, modules: restructured source code tree

- new folder src/ to hold the source code for main project applications
- main.c is in src/
- all core files are subfolder are in src/core/
- modules are in src/modules/
- libs are in src/lib/
- application Makefiles are in src/
- application binary is built in src/ (src/kamailio)

Daniel-Constantin Mierla authored on 07/12/2016 11:03:51
Showing 1 changed files
1 1
new file mode 100644
... ...
@@ -0,0 +1,287 @@
1
+DB Cassandra Module
2
+
3
+Anca Vamanu
4
+
5
+   <anca.vamanu@1and1.ro>
6
+
7
+Boudewyn Ligthart
8
+
9
+   <bligthart@btlnet.co.uk>
10
+
11
+Edited by
12
+
13
+Anca Vamanu
14
+
15
+   <anca.vamanu@1and1.ro>
16
+
17
+   Copyright © 2012 1&1 Internet AG
18
+     __________________________________________________________________
19
+
20
+   Table of Contents
21
+
22
+   1. Admin Guide
23
+
24
+        1. Overview
25
+        2. Dependencies
26
+
27
+              2.1. SIP Router Modules
28
+              2.2. External Libraries or Applications
29
+
30
+        3. Parameters
31
+
32
+              3.1. schema_path (string)
33
+
34
+        4. Functions
35
+        5. Installation
36
+        6. Table schema
37
+        7. Limitations
38
+
39
+   List of Examples
40
+
41
+   1.1. Set schema_path parameter
42
+
43
+Chapter 1. Admin Guide
44
+
45
+   Table of Contents
46
+
47
+   1. Overview
48
+   2. Dependencies
49
+
50
+        2.1. SIP Router Modules
51
+        2.2. External Libraries or Applications
52
+
53
+   3. Parameters
54
+
55
+        3.1. schema_path (string)
56
+
57
+   4. Functions
58
+   5. Installation
59
+   6. Table schema
60
+   7. Limitations
61
+
62
+1. Overview
63
+
64
+   Db_cassandra is one of the Kamailio database modules. It does not
65
+   export any functions executable from the configuration scripts, but it
66
+   exports a subset of functions using the database API, and thus, other
67
+   modules can use it as a database driver, instead of, for example, the
68
+   Mysql module.
69
+
70
+   The storage backend is a Cassandra cluster and this module provides an
71
+   SQL interface to be used by other modules for storing and retrieving
72
+   data. Because Cassandra is a NoSQL distributed system, there are
73
+   limitations on the operations that can be performed. The limitations
74
+   concern the indexes on which queries are performed, as it is only
75
+   possible to have simple conditions (equality comparison only) and only
76
+   two indexing levels. These issues will be explained in an example
77
+   below.
78
+
79
+   Cassandra DB is especially suited for storing large amounts of data or
80
+   data that requires distribution, redundancy or replication. One usage
81
+   example is a distributed location system in a platform that has a
82
+   cluster of SIP Router servers, with several proxies and registration
83
+   servers accessing the same location database. This was actually the
84
+   main use case we had in mind when implementing this module. Please NOTE
85
+   that it has only been tested with the usrloc, auth_db and domain
86
+   modules.
87
+
88
+   You can find a configuration file example for this usage in the module
89
+   - kamailio_cassa.cfg.
90
+
91
+   Because the module has to do the translation from SQL to Cassandra
92
+   NoSQL queries, the schemas for the tables must be known by the module.
93
+   You will find the schemas for location, subscriber and version tables
94
+   in utils/kamctl/dbcassandra directory. You have to provide the path to
95
+   the directory containing the table definitions by setting the module
96
+   parameter schema_path.
97
+
98
+   There is no need to configure a table metadata in Cassandra cluster.
99
+   You only need to define a keyspace with the name of the database and
100
+   for each table a column family inside that keyspace with the name of
101
+   the table. The comparator and validators should be either UTF8Type or
102
+   ASCIIType. Example:
103
+   ...
104
+   create keyspace kamailio;
105
+   use kamailio;
106
+   create column family 'location' with comparator='UTF8Type' and
107
+default_validation_class='UTF8Type' and key_validation_class='UTF8Type';
108
+   ...
109
+
110
+   Special attention was given to performance in Cassandra. Therefore, the
111
+   implementation uses only the native row indexing in Cassandra and no
112
+   secondary indexes, because they are costly. Instead, we simulate a
113
+   secondary index by using the column names and putting information in
114
+   them, which is very efficient. Also, for deleting expired records, we
115
+   let Cassandra take care of this with its own mechanism (by setting the
116
+   TTL for columns).
117
+
118
+   The module supports raw queries. However these queries must follow the
119
+   CQL (Cassandra Query Language) syntax. The queries can be issued in the
120
+   script by means of the AVPOPS module. Keep in mind that when passing
121
+   back the results from the database only the first row is used to set
122
+   the AVP variables. (default AVPOPS behaviour) The script lines below
123
+   can be used as an example for issuing the query towards an cassandra
124
+   instance.(This example will work once the column family `location` is
125
+   configured correctly in the cassandra keyspace)
126
+   ...
127
+   $var(dballowed)="select * from location where key = 'userx' limit 1;";
128
+   avp_db_query("$var(dballowed)");
129
+   xlog("L_INFO","Got result here: [$avp(i:1)] [$avp(i:2)] [$avp(i:3)].\n");
130
+   ...
131
+
132
+2. Dependencies
133
+
134
+   2.1. SIP Router Modules
135
+   2.2. External Libraries or Applications
136
+
137
+2.1. SIP Router Modules
138
+
139
+   The following modules must be loaded before this module:
140
+     * No dependencies on other SIP Router modules.
141
+
142
+2.2. External Libraries or Applications
143
+
144
+   The following libraries or applications must be installed before
145
+   running SIP Router with this module loaded:
146
+     * Thrift library (tested with version 0.6.1 and version 0.7.0). You
147
+       can download it from http://archive.apache.org/dist/thrift .
148
+
149
+   The implementation was tested with Cassandra version 1.0.1 and version
150
+   1.1.6. I used the sourced from DataStax Community Edition
151
+   (http://www.datastax.com/download/community).
152
+
153
+3. Parameters
154
+
155
+   3.1. schema_path (string)
156
+
157
+3.1. schema_path (string)
158
+
159
+   The directory where the files with the table schemas are located. This
160
+   directory has to contain the subdirectories corresponding to the
161
+   database name (name of the directory = name of the database). These
162
+   directories, in turn, contain the files with the table schemas. See the
163
+   schemas in utils/kamctl/dbcassandra directory.
164
+
165
+   Example 1.1. Set schema_path parameter
166
+   ...
167
+   modparam("db_cassandra", "schema_path",
168
+               "/usr/local/kamailio/etc/kamctl/dbcassandra")
169
+   ...
170
+
171
+4. Functions
172
+
173
+   NONE
174
+
175
+5. Installation
176
+
177
+   Because the db_cassandra module dependes on an external library, it is
178
+   not compiled and installed by default. You can use one of these
179
+   options:
180
+     * - edit the "Makefile" and remove "db_cassandra" from
181
+       "excluded_modules" list. Then follow the standard procedure to
182
+       install SIP Router: "make all; make install".
183
+     * - from command line, run: 'make all include_modules="db_cassandra";
184
+       make install include_modules="db_cassandra"'.
185
+
186
+6. Table schema
187
+
188
+   The module must know the table schema for the tables that will be used.
189
+   You must configure the path to the schema directory by setting the
190
+   schema_path parameter.
191
+
192
+   A table schema document has the following structure:
193
+     * First row: the name and type of the columns in the form name(type)
194
+       separated by spaces. The possible types are: string, int, double
195
+       and timestamp.
196
+       Thetimestamp type has a special meaning. Only one column of this
197
+       type can be defined for a table, and it should contain the expiry
198
+       time for that record. If defined this value will be used to compute
199
+       the ttl for the columns and Cassandra will automatically delete the
200
+       columns when they expire. Because we want the ttl to have meaning
201
+       for the entire record, we must ensure that when the ttl is updated,
202
+       it is updated for all columns for that record. In other words, to
203
+       update the expiration time of a record, an insert operation must be
204
+       performed from the point of view of the db_cassandra module
205
+       ("insert" in Cassandra means "replace if exists or insert new
206
+       record otherwise"). So, if you define a table with a timestamp
207
+       column, the update operations on that table that also update the
208
+       timestamp must update all columns. So, these update operations must
209
+       in fact be insert operations.
210
+     * Second row: the columns that form the row key separated by space.
211
+     * Third row: the columns that form the secondary key separated by
212
+       space.
213
+
214
+   Below you can see the schema for the location table (when use_domain
215
+   not set):
216
+
217
+   ...
218
+   callid(string) cflags(int) contact(string) cseq(int) expires(timestamp) flags
219
+(int) last_modified(int) methods(int) path(string) q(double) received(string) so
220
+cket(string) user_agent(string) username(string) ruid(string) instance(string) r
221
+eg_id(int)
222
+   username
223
+   contact
224
+   ...
225
+
226
+   Observe first that the row key is the username and the secondary index
227
+   is the contact. We have also defined a timestamp column - expires.
228
+
229
+   If you need to use the domain part of the AOR also (you have set
230
+   use_domain parameter for usrloc in the script), you should include the
231
+   domain column in the list of columns and in the primary key. The schema
232
+   will then look like this:
233
+   ...
234
+   callid(string) cflags(int) contact(string) cseq(int) domain(string) expires(t
235
+imestamp) flags(int) last_modified(int) methods(int) path(string) q(double) rece
236
+ived(string) socket(string) user_agent(string) username(string) ruid(string) ins
237
+tance(string) reg_id(int)
238
+   username domain
239
+   contact
240
+   ...
241
+
242
+   Notice that a key (primary or secondary) can be composed from more
243
+   columns, in which case you have to specify them separated by space.
244
+
245
+   To understand why the schema looks like this, we must first see which
246
+   queries are performed on the location table. (The 'callid' condition
247
+   was ignored as it doesn't really have a well defined role in the SIP
248
+   RFC).
249
+     * When Invite received, lookup location: select where username='..'.
250
+     * When Register received, update registration: update where
251
+       username='..' and contact='..'.
252
+
253
+   So, the relation between these keys is the following:
254
+     * The unique key for a table is actually the combination of row key +
255
+       secondary key.
256
+     * A row defined by a row key will contain more records with different
257
+       secondary keys.
258
+
259
+   The timestamp column that leaves the Cassandra cluster to deal with
260
+   deleting the expired records. For this to work right we needed to
261
+   modify a bit the behavior of usrloc module and replace update sql query
262
+   performed at re-registration with an insert sql query (so that all
263
+   columns are updated and the new timestamp is set for all columns). This
264
+   behavior is enabled by setting a parameter in the usrloc module
265
+   db_update_as_insert:
266
+
267
+   ...
268
+   modparam("usrloc", "db_update_as_insert", 1)
269
+   ...
270
+
271
+   Also you should disable in usrloc module the timer routine that checks
272
+   for expired records. You can do this by setting the timer interval to
273
+   0. timer_interval:
274
+
275
+   ...
276
+   modparam("usrloc", "timer_interval", 0)
277
+   ...
278
+
279
+   The alternative would have been to define an index on the expire column
280
+   and run a external job to periodically delete the expired records.
281
+   However, obviously, this would be more costly.
282
+
283
+7. Limitations
284
+
285
+   The module can be used only when the queries use only one index, which
286
+   is also the unique key, or have two indexes that form the unique key
287
+   like in the usrloc usage.