doc/tcp_tunning.txt
736a8194
 Kamailio TCP Tunning/monitoring for lots of open connections
b7e2b56c
 ==============================================================
408ed449
 
 0. Introduction
 ----------------
 
 This document describes very briefly various settings that should improve
736a8194
 Kamailio TCP performance for sites handling a lot of TCP traffic (> 1000 open
46e13e40
 connections or very high connection/disconnection rates).
408ed449
 
46e13e40
 For now it deals only with Linux specific optimizations.
408ed449
 
 
b7e2b56c
 1. Useful Linux kernel settings
 -------------------------------
46e13e40
 
 1.1 Connection rate/pending connections: by default the connection rate is
408ed449
  too small
 
 net.core.somaxconn             -  limit of the listen() backlog, default 128
 net.ipv4.tcp_max_syn_backlog   -  default 1024 or 128
 net.ipv4.tcp_timestamps        - default on., should be on (along with 
                                  tcp_tw_recycle and timestamp supporting
                                  peers allows for fast connections rates)
 
46e13e40
 1.2 connection in close_wait
 
  Connection should stay as little as possible
408ed449
  in close_wait to quickly free the fd/resources for new connections attempts
b7e2b56c
  WARNING: this could break normal TCP use, use it only if you know what you are
408ed449
   doing
  
 net.ipv4.tcp_max_tw_buckets - maximum number of timewait sockets
                               (the default seems to be ok)
 net.ipv4.tcp_tw_recycle     - enables fast time wait sockets recycling (default
                               off), should be enabled if you have lots of short
                               lived connections 
                               WARNING: see the above warning
 net.ipv4.tcp_tw_reuse       - allows reusing of time-wait sockets (default off)
                               WARNING: see above
 
 net.ipv4.tcp_syncookies     - default off, in this case it's probably better to
                               keep it off
 
46e13e40
 1.3 Port range
b7e2b56c
 
408ed449
 net.ipv4.ip_local_port_range - should be increased (e.g. 4096-65534)
 
46e13e40
 1.4 Open file descriptors
 
408ed449
 fs.file-max                 - maximum number of fds that will be allocated
                               (you probably need to increase it, default 
                                depends on installed memory)
 
 1.5 other sysctl that might affect tcp connection rate or the maximum number
     of open connections
 
67e3e42e
 fs.epoll.max_user_instances - maximum number of devices - per user (2.6.27.8+)
 fs.epoll.max_user_watches   - maximum number of "watched" fds - per user
                               (2.6.27.8+)
 
408ed449
 net.ipv4.tcp_max_orphans    - might be worth a look if things go wrong
 net.core.netdev_max_backlog - maximum device backlog
 
46e13e40
 Related applications
 --------------------
408ed449
 ifconfig <dev> txqueuelen <val> - set device transmission queue len
 
46e13e40
 iptables  - remove the ip_conntrack module (it limits the maximum tcp
   connections, adds extra overhead (slow)). It's probably better to remove
   all the iptables modules.
 
 
408ed449
 2. Monitoring (values to watch for)
 -----------------------------------
 
46e13e40
 2.1 File descriptors
408ed449
 
 fs.dentry-state  - format: nr. dentries, nr. unused, age_limit, want_pages
 fs.file-nr       - format: allocated, unused, max (==fs.file-max)
 fs.inode-state  -  format: nr. allocated, nr. free, preshrink
 
 2.2 TCP
 
 /proc/net/netstat - the TW, TWRecycled, TWKilled, PAWPassive, PAWActive,
                     PASWEstab fields
                    ( cat /proc/net/netstat |cut -d" " -f12-17 ;
                      cat /proc/net/sockstat)
 /proc/net/sockstat
 
 
736a8194
 3. Kamailio settings
46e13e40
 ----------------------
408ed449
 
46e13e40
 - Don't forget to increase tcp_max_connections and the amount of shared memory
736a8194
 - You should increase the number of Kamailio "tcp_children" processes (-N no)
46e13e40
   As a rule of thumb, (maximum simultaneous connections)/2000 should be ok
 - You might have to decrease TCP_BUF_SIZE to a smaller value (e.g 8K)
 - You might want to increase PKG_MEM_POOL_SIZE (for large queues)
408ed449
 
736a8194
 - You might need to increase the maximum open fds limit before starting Kamailio
46e13e40
   (e.g. ulimit -n 1000000)
408ed449