HTTP_ASYNC_CLIENT Module

Federico Cabiddu

   <federico.cabiddu@gmail.com>

Giacomo Vacca

   <giacomo.vacca@gmail.com>

Camille Oudot

   Orange
   <camille.oudot@orange.com>

Edited by

Federico Cabiddu

   <federico.cabiddu@gmail.com>

   Copyright © 2016 Federico Cabiddu
     __________________________________________________________________

   Table of Contents

   1. Admin Guide

        1. Overview
        2. Dependencies

              2.1. Kamailio Modules
              2.2. External Libraries or Applications

        3. Parameters

              3.1. workers (integer)
              3.2. connection_timeout (integer)
              3.3. hash_size (integer)
              3.4. tls_version (integer)
              3.5. tls_verify_host (integer)
              3.6. tls_verify_peer (integer)
              3.7. curl_verbose (integer)
              3.8. memory_manager (string)
              3.9. tls_client_cert (string)
              3.10. tls_client_key (string)
              3.11. tls_ca_path (string)
              3.12. tcp_keepalive (integer)
              3.13. tcp_ka_idle (integer)
              3.14. tcp_ka_interval (integer)

        4. Functions

              4.1. http_async_query(url, route_name)

        5. Pseudo Variables
        6. Statistics

              6.1. requests
              6.2. replies
              6.3. errors
              6.4. timeouts

        7. Remarks

   List of Examples

   1.1. Set workers parameter
   1.2. Set connection_timeout parameter
   1.3. Set hash_size parameter
   1.4. Set tls_version parameter
   1.5. Set tls_verify_host parameter
   1.6. Set tls_verify_peer parameter
   1.7. Set curl_verbose parameter
   1.8. Set memory_manager parameter
   1.9. Set tls_client_cert parameter
   1.10. Set tls_client_key parameter
   1.11. Set tls_ca_path parameter
   1.12. Set tcp_keepalive parameter
   1.13. Set tcp_ka_idle parameter
   1.14. Set tcp_ka_interval parameter
   1.15. http_async_query() usage
   1.16. $http_req_id variable usage
   1.17. $http_req(key) variable usage

Chapter 1. Admin Guide

   Table of Contents

   1. Overview
   2. Dependencies

        2.1. Kamailio Modules
        2.2. External Libraries or Applications

   3. Parameters

        3.1. workers (integer)
        3.2. connection_timeout (integer)
        3.3. hash_size (integer)
        3.4. tls_version (integer)
        3.5. tls_verify_host (integer)
        3.6. tls_verify_peer (integer)
        3.7. curl_verbose (integer)
        3.8. memory_manager (string)
        3.9. tls_client_cert (string)
        3.10. tls_client_key (string)
        3.11. tls_ca_path (string)
        3.12. tcp_keepalive (integer)
        3.13. tcp_ka_idle (integer)
        3.14. tcp_ka_interval (integer)

   4. Functions

        4.1. http_async_query(url, route_name)

   5. Pseudo Variables
   6. Statistics

        6.1. requests
        6.2. replies
        6.3. errors
        6.4. timeouts

   7. Remarks

1. Overview

   This module performs asynchronous HTTP queries.

2. Dependencies

   2.1. Kamailio Modules
   2.2. External Libraries or Applications

2.1. Kamailio Modules

   The following modules must be loaded before this module:
     * tm - Transaction module
     * pv - Pseudo-Variables module

2.2. External Libraries or Applications

   The following libraries or applications must be installed before
   running Kamailio with this module loaded:
     * libcurl libev

3. Parameters

   3.1. workers (integer)
   3.2. connection_timeout (integer)
   3.3. hash_size (integer)
   3.4. tls_version (integer)
   3.5. tls_verify_host (integer)
   3.6. tls_verify_peer (integer)
   3.7. curl_verbose (integer)
   3.8. memory_manager (string)
   3.9. tls_client_cert (string)
   3.10. tls_client_key (string)
   3.11. tls_ca_path (string)
   3.12. tcp_keepalive (integer)
   3.13. tcp_ka_idle (integer)
   3.14. tcp_ka_interval (integer)

3.1. workers (integer)

   Number of worker processes to be started to send HTTP requests and
   asynchronously handle responses.

   Default value is 1.

   Example 1.1. Set workers parameter
...
modparam("http_async_client", "workers", 2)
...

3.2. connection_timeout (integer)

   Defines in milliseconds how long Kamailio waits for a response from
   HTTP server.

   Default value is 500ms.

   Example 1.2. Set connection_timeout parameter
...
modparam("http_async_client", "connection_timeout", 1000)
...

3.3. hash_size (integer)

   The size of the hash table internally used to keep the requests. A
   larger table is much faster but consumes more memory. The hash size
   must be a power of two, otherwise it will be rounded down to the
   nearest power of two.

   Default value is 2048.

   Example 1.3. Set hash_size parameter
...
modparam("http_async_client", "hash_size", 1024)
...

3.4. tls_version (integer)

   For HTTPS connections, what's the preferred SSL version.
   http://curl.haxx.se/libcurl/c/CURLOPT_SSLVERSION.html

   Default value is 0 (default SSL version).

   Example 1.4. Set tls_version parameter
...
modparam("http_async_client", "tls_version", 6)
...

3.5. tls_verify_host (integer)

   For HTTPS connections, whether the client should verify the server
   host. http://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYHOST.html

   Default value is 2 (enabled).

   Example 1.5. Set tls_verify_host parameter
...
modparam("http_async_client", "tls_verify_host", 0)
...

3.6. tls_verify_peer (integer)

   For HTTPS connections, whether the client should verify the server
   identity. http://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYPEER.html

   Default value is 1 (enabled).

   Example 1.6. Set tls_verify_peer parameter
...
modparam("http_async_client", "tls_verify_peer", 0)
...

3.7. curl_verbose (integer)

   If defined to a non-zero value, extra information from cURL (request
   and response headers) will be included in the kamailio logs, with
   LM_INFO priority.

   Default value is 0 (disabled).

   Example 1.7. Set curl_verbose parameter
...
modparam("http_async_client", "curl_verbose", 1)
...

3.8. memory_manager (string)

   Choose the memory manager used by curl:
     * shm: curl will use kamailio's SHM pool and memory manager
     * sys: curl will use the system memory and memory manager (malloc,
       free, ...)

   Note: if this module is used in conjunction with another module using
   libcurl (http_client, utils, xcap, xcap_client), it must be loaded as
   first one for this parameter to have effect, otherwise curl will likely
   use the system memory allocator by default. On the other hand if the
   module is loaded before any other module using libcurl, all the modules
   will use the memory manager specified by this parameter.

   Default value "shm"

   Example 1.8. Set memory_manager parameter
...
modparam("http_async_client", "memory_manager", "sys")
...

3.9. tls_client_cert (string)

   For HTTPS connections, the file path of the TLS client certificate to
   be used. http://curl.haxx.se/libcurl/c/CURLOPT_SSLCERT.html

   Default value is NULL (not used). Default type is PEM.

   Example 1.9. Set tls_client_cert parameter
...
modparam("http_async_client", "tls_client_cert", "/etc/kamailio/ssl/clientcert.p
em")
...

3.10. tls_client_key (string)

   For HTTPS connections, the file path of the TLS client certificate key.
   http://curl.haxx.se/libcurl/c/CURLOPT_SSLKEY.html

   Default value is NULL (not used).

   Example 1.10. Set tls_client_key parameter
...
modparam("http_async_client", "tls_client_key", "/etc/kamailio/ssl/clientcert.ke
y")
...

3.11. tls_ca_path (string)

   The path of the directory holding the CA certificates.
   http://curl.haxx.se/libcurl/c/CURLOPT_CAPATH.html

   Default value is NULL..

   Example 1.11. Set tls_ca_path parameter
...
modparam("http_async_client", "tls_ca_path", "/etc/kamailio/ssl/ca/")
...

3.12. tcp_keepalive (integer)

   If defined to a non-zero value, TCP keepalive will be enabled on cURL
   connections.

   Requires libcurl >= 7.25.0

   Default value is 0 (disabled).

   Example 1.12. Set tcp_keepalive parameter
...
modparam("http_async_client", "tcp_keepalive", 1)
...

3.13. tcp_ka_idle (integer)

   TCP keep-alive idle time wait value (in seconds).

   Default value is 0 (use cURL default value, 60s).

   Example 1.13. Set tcp_ka_idle parameter
...
modparam("http_async_client", "tcp_ka_idle", 30)
...

3.14. tcp_ka_interval (integer)

   TCP keep-alive interval (in seconds).

   Default value is 0 (use cURL default value, 60s).

   Example 1.14. Set tcp_ka_interval parameter
...
modparam("http_async_client", "tcp_ka_interval", 120)
...

4. Functions

   4.1. http_async_query(url, route_name)

4.1.  http_async_query(url, route_name)

   Sends HTTP(S) request asynchronously to the URL given in “url”
   parameter, which is a string that may contain pseudo variables.

   Parameter “route_name” defines the route to be executed upon reception
   of HTTP reply, on error or on timeout. If a transaction exists before
   calling http_async_query(), it will be paused and resumed in this
   route, while the routing script execution will be stopped. If executed
   in a transactionless context, or if $http_req(suspend) is used not to
   suspend the transaction, the routing script execution will continue and
   the query result will be available in “route_name”.

   Return value: 0 (stop script execution) on success in transaction
   context, 1 (continue script execution) in transaction-less context (or
   if $http_req(suspend) is used), -1 on error.

   This function can be used from ANY_ROUTE.

   This method is executed asynchronously. The HTTP return code, body and
   error are returned in the module-specific $http_* PVs (see below). See
   example on how to retrieve return values.

   Example 1.15. http_async_query() usage
...
# create a transaction to be paused, and resumed in route[HTTP_REPLY]
t_newtran();
# GET
http_async_query("http://example.com/test.php?r_uri=$rU&f_uri=$fU", "HTTP_REPLY"
);
...
# POST
$http_req(body) = "{'r_uri':'" + $rU + "', 'f_uri':'" + $fU + "'}";
http_async_query("http://example.com/test.php", "HTTP_REPLY");
...
route[HTTP_REPLY] {
    if ($http_ok) {
        xlog("L_INFO", "route[HTTP_REPLY]: status $http_rs\n");
        xlog("L_INFO", "route[HTTP_REPLY]: body   $http_rb\n");
    } else {
        xlog("L_INFO", "route[HTTP_REPLY]: error  $http_err)\n");
    }
}
...

5. Pseudo Variables

   The $http_req_id read-only variable can be used in REQUEST_ROUTE to
   retrive the unique identifier for a query after sending it or in the
   HTTP callback route to retrive the id of the query the reply belongs
   to. Useful mainly in non-transactional context.

   Example 1.16. $http_req_id variable usage
...
$http_req(suspend) = 0;

http_async_query("https://example.com/test.php", "HTTP_REPLY");

xlog("L_INFO", "Query id: $http_req_id");
...
route[HTTP_REPLY] {
xlog("L_INFO", "received reply for query $http_req_id\n");
}
...

   The $http_req(key) write-only variable can be used to set custom
   parameters before sending a HTTP query

   key can be one of:
     * all: if set to $null, resets all the parameters to their default
       value (the ones defined in modparam)
     * hdr: sets/modifies/removes a HTTP header. N.B.: setting this
       variable multiple times will add several headers to the query.
     * body: sets/modifies/removes the request body
     * method: sets the HTTP method: either "GET", "POST", "PUT" or
       "DELETE" (these are the supported methods). (Note: if the method is
       not set, curl will use GET, or POST if a body is specified)
     * timeout: sets the HTTP timeout. (Note, this timeout should be
       normally less than tm.fr_timer timeout, because transaction timeout
       has a higher priority over HTTP timeout)
     * tls_client_cert: sets the client certificate to use
     * tls_client_key: sets the client certificate key to use
     * tls_ca_path: sets the CA certificate path to use
     * authmethod: Sets the preferred authentication mode for HTTP/HTTPS
       requests. The value is a bitmap and multiple methods can be used.
       Note that in this case, the CURL library will make an extra request
       to discover server-supported authentication methods. You may want
       to use a specific value.
       Valid values are:
          + 1 - BASIC authentication
          + 2 - HTTP Digest authentication
          + 4 - GSS-Negotiate authentication
          + 8 - NTLM authentication
          + 16 - HTTP Digest with IE flavour
       Default value is 3 - BASIC and Digest authentication.
     * username: sets the username to use for authenticated requests
     * password: sets the password to use for authenticated requests
     * suspend: if set to 0 it doesn't suspend the current transaction
       before performing the query
     * tcp_keepalive: enable TCP keepalive
     * tcp_ka_idle: set TCP keepalive idle time wait
     * tcp_ka_interval: set TCP keepalive interval

   Example 1.17. $http_req(key) variable usage
...
$http_req(all) = $null;                    # reset the parameters
$http_req(timeout) = 100;                  # 100 ms
$http_req(method) = "DELETE";
$http_req(hdr) = "X-Sip-Call-Id: " + $ci;
$http_req(hdr) = "X-Foo: bar";             # add a 2nd header
$http_req(suspend) = 0;             # don't suspend the transaction, continue ro
uting script's execution
# the following request will use the above parameters
http_async_query("https://example.com/test.php", "HTTP_REPLY");
...

   The following read-only pseudo variables can only be used in the
   callback routes executed by http_async_query()
     * $http_ok: 1 if cURL executed the request successfully, 0 otherwise
       (check $http_err for details)
     * $http_err: cURL error string if an error occurred, $null otherwise
     * $http_rs: http status
     * $http_rr: http reason phrase
     * $http_hdr(Name): value of the Name header (the $(http_hdr(Name)[N])
       syntax can also be used, check the SIP $hdr() PV documentation for
       details)
     * $http_mb and $http_ml: HTTP response buffer (including headers) and
       length
     * $http_rb and $http_bs: HTTP response body and body length
     * $http_time: provides access to broken-down transfer time in
       microseconds calculated by cURL
       (https://curl.se/libcurl/c/curl_easy_getinfo.html).
       'name' can be:
          + total - total time for the transfer
          + lookup - name lookup time
          + connect - time from the start until the connection to the
            remote host (or proxy) was completed.
          + appconnect - time from start until SSL/SSH handshake completed
          + pretransfer - time from start until just before the transfer
            begins
          + starttransfer - time from start until just when the first byte
            is received
          + redirect - time taken for all redirect steps before the final
            transfer.

6. Statistics

   6.1. requests
   6.2. replies
   6.3. errors
   6.4. timeouts

6.1. requests

   The number of http requests sent.

6.2. replies

   The number of received http replies.

6.3. errors

   The number of errors.

6.4. timeouts

   The number of timed out requests.

7. Remarks

   Note: libcurl leak in CentOS 6 - this module uses libcurl library and
   in case if you are using CentOS 6, be aware that standard
   libcurl-7.19.7-52 has a memory leak. To fix this memory, install
   libcurl from city-fan repository. More details at:
   https://www.digitalocean.com/community/questions/how-to-upgrade-curl-in
   -centos6