Linux settings for high load
systems
There are a few basic settings
you have to adjust for high load systems to make sure the server have
enough resources to handle big number of network connections. The main
parameter is a maximum number of opened files allowed for the process to
keep at the same time. Each network connection uses a file handler
therefore if the limit is too low you can quickly run out of handlers
and the server can not accept any more connections.
This limit is set on
2 levels - on the kernel level (fs.file-max) and on the system level
(nofile). Another kernel property which can be important in certain
configurations (like transports installations or when you use proxy for
Bosh connections) is: net.ipv4.ip_local_port_range. This parameter can
be set the same way as the fs.file-max property.
fs.file-max
The fs.file-max kernel
property is set via sysctl command. You can see current settings
executing command:
view source print?
1 # sysctl fs.file-max
2 fs.file-max = 358920
If you plan to run high load
service with big number of server connections then this parameter should
be at least as twice big as the number of network connections you
expect to support. You can change this setting executing command:
view source
print?
1 # sysctl -w
fs.file-max=360000
2 fs.file-max = 360000
net.ipv4.ip_local_port_range
You can see current settings
executing command:
view source
print?
1 # sysctl
net.ipv4.ip_local_port_range
2 net.ipv4.ip_local_port_range
= 32768 61000
You can change this setting
executing command:
view source
print?
1 # sysctl -w
net.ipv4.ip_local_port_range="1024 65000"
2 net.ipv4.ip_local_port_range
= 1024 65000
/etc/sysctl.conf
Above commands let the system
remember new settings until the next system restart. If you want to make
the change permanent you have to edit file: /etc/sysctl.conf and add
the property at the end of the file:
view source
print?
1 fs.file-max=360000
2
net.ipv4.ip_local_port_range=1024 65000
It will be automatically
loaded next time you start the server. Command:
view source
print?
1 # sysctl -p
Causes the /etc/systcl.conf to
be reloaded which is useful when you added more parameters to the file
and don't want to restart the server.
nofile
This is the property used by
the system limits. For example running the command ulimit -a shows you
all limits set for the current user:
view source
print?
01 # ulimit -a
02 core file size
(blocks, -c) 0
03 data seg size
(kbytes, -d) unlimited
04 file size
(blocks, -f) unlimited
05 pending signals
(-i) 38912
06 max locked memory
(kbytes, -l) 32
07 max memory size
(kbytes, -m) unlimited
08 open files
(-n) 40960
09 pipe size (512
bytes, -p) 8
10 POSIX message queues
(bytes, -q) 819200
11 stack size
(kbytes, -s) 8192
12 cpu time
(seconds, -t) unlimited
13 max user processes
(-u) 38912
14 virtual memory
(kbytes, -v) unlimited
15 file locks
(-x) unlimited
To make it even more
interesting and more complex there are 2 types of system limits: soft
limit which can be temporarily exceeded by the user and hard limit which
can not be exceeded. To see your hard limit execute command:
view source
print?
01 # ulimit -a -H
02 core file size
(blocks, -c) unlimited
03 data seg size
(kbytes, -d) unlimited
04 file size
(blocks, -f) unlimited
05 pending signals
(-i) 38912
06 max locked memory
(kbytes, -l) 32
07 max memory size
(kbytes, -m) unlimited
08 open files
(-n) 40960
09 pipe size (512
bytes, -p) 8
10 POSIX message queues
(bytes, -q) 819200
11 stack size
(kbytes, -s) unlimited
12 cpu time
(seconds, -t) unlimited
13 max user processes
(-u) 38912
14 virtual memory
(kbytes, -v) unlimited
15 file locks
(-x) unlimited
The hard limits are usually
bigger then the soft limits or sometimes the same. For us the most
important parameter is: open files. You can change the property in file:
/etc/security/limits.conf. You have to append 2 following lines to the
end of the file:
view source
print?
1 jabber soft
nofile 350000
2 jabber hard
nofile 350000
Where the jabber is the user
name of the account running you IM service. You can also set the limits
for all users on the machine in a following way:
view source
print?
1 * soft
nofile 350000
2 * hard
nofile 350000
For those changes to make an
effect you have to logout from the modified account and login again. New
limits should be applied.
No comments:
Post a Comment