Tuning the System
This chapter contains recommendations on how to tune a Paragon Active Assurance system for optimum performance.
CPU
BIOS
Processors typically provide settings to enable, disable, and tune processor-level features. Today's systems usually provide maximum performance by default and do not need to be adjusted. However, we recommend that you ensure Turbo Boost is enabled in order to achieve slightly higher performance.
Scaling Governors
Linux supports different CPU scaling profiles (powersave,
performance) that control the CPU clock frequencies via the kernel. Use
the performance profile to achieve the best performance. To ensure that
the maximum clock frequency is always used, you need to set this for all CPUs. You can
find out the number of CPUs with the nproc command. Then run the
following for each CPU X:
echo performance > /sys/devices/system/cpu/cpufreq/policyX/scaling_governor
Note that setting the CPUs to run at maximum frequency is applicable to hardware systems only and may come with added costs.
Memory
The swappiness Linux kernel parameter controls how much (and how often) the Linux
kernel will copy RAM contents to the swap space.
We recommend setting the amount of swapping to a minimum, without disabling it entirely, so that the memory pages are kept in physical memory:
sudo sysctl -w vm.swappiness=1
(The sysctl command is omitted from the kernel parameter settings that
follow below.)
Storage
Partitions and File Systems
This is addressed in the Installation Guide, chapter Installing Required OS and Software.
In addition, page cache flushing may be tuned to provide a more even behavior: background flush earlier, aggressive flush later.
vm.dirty_ratio = 80 vm.dirty_background_ratio = 5 vm.dirty_expire_centisecs = 12000
Read-ahead Size
The tunable read_ahead_kb parameter for storage block devices defines the
maximum number of kilobytes that the operating system may read ahead during a sequential read
operation. As a result, the likely-needed information is already present within the kernel page
cache for the next sequential read, which improves read I/O performance.
We recommend setting this parameter to 4096 KB for all block devices:
echo 4096 > /sys/block/sdN/queue/read_ahead_kb
Network
Socket and TCP Buffers
The maximum socket buffer size (in bytes) should be set as follows for all protocol types and for both reads and writes to support full-duplex 10GbE connections:
net.core.rmem_max = 16777216 net.core.wmem_max = 16777216
To improve TCP throughput, set the auto-tuning parameters for the TCP read and write buffers: the minimum, default, and maximum number of bytes to use. We recommend the settings below. Increasing the minimum and default will consume more memory per connection, which may not be necessary.
net.ipv4.tcp_wmem = 4096 349520 16777216 net.ipv4.tcp_rmem = 4096 349520 16777216
TCP Backlog
The following TCP settings should be increased from their defaults to better handle bursts of load.
TCP backlog queue for half-open connections:
net.ipv4.tcp_max_syn_backlog = 8192
TCP backlog queue for passing connections to accept:
net.core.somaxconn = 4096
Device Backlog
The length of the network device backlog queue per CPU should be increased from the default:
net.core.netdev_max_backlog = 1000
Local Port Range
Extend the range of IP ports that are allowed for TCP and UDP connections to provide enough ephemeral ports for the anticipated workload:
net.ipv4.ip_local_port_range = 10240 65535
Apache
Performance
The following settings are recommended in
/etc/apache2/conf-available/performance.conf:
HostnameLookups Off KeepAlive On KeepAliveTimeout 2 MaxKeepAliveRequests 600 Timeout 120
Event Multi-processing
The following settings are recommended in
/etc/apache2/mods-enabled/mpm_event.conf:
<IfModule mpm_event_module>
StartServers 3
ServerLimit 8
MinSpareThreads 8
MaxSpareThreads 75
ThreadLimit 64
ThreadsPerChild 25
MaxRequestWorkers 200
MaxConnectionsPerChild 10000
</IfModule>Security
The following settings are recommended in
/etc/apache2/conf-available/security.conf:
ServerTokens Prod ServerSignature Off TraceEnable Off
Enabling the Configurations
Make sure that all Apache configurations above are saved, then enable them as follows:
a2enconf performance a2enconf security a2enmod mpm_event systemctl reload apache2
OpenVPN
Transmit Queue Length
To improve OpenVPN performance and throughput, it is advisable to adjust the transmit queue
length for the tun interface:
echo "txqueuelen 2000" >> /etc/openvpn/netrounds.conf systemctl restart openvpn@netrounds.service
PostgreSQL
The following settings are recommended in
/etc/postgresql/10/main/postgresql.conf for best performance:
max_connections = 100 effective_io_concurrency = 128 # between 64 and 256 based on performance of underlying storage stack default_statistics_target = 100 checkpoint_completion_target = 0.9 random_page_cost = 1.1 max_worker_processes = 8 max_parallel_workers_per_gather = 1 max_parallel_workers = 8 shared_buffers = X # set to 25% of available memory effective_cache_size = work_mem = maintenance_work_mem = temp_buffers = 16MB wal_buffers = 16MB 18wal_compression = on
Unattended Software Upgrades
To avoid automatically installing software packages, we advise you to disable the
unattended-upgrades service or to remove it entirely from the system:
systemctl stop unattended-upgrades systemctl disable unattended-upgrades
Control Center
Finally, a number of Control Center parameters can be tuned according to the available resources and the system load. These include:
/etc/netrounds/netrounds.confCALL_EXECUTER_MAX_CHILDREN: This configures how many background jobs can run in parallel. The default is 20. Each test and monitor (not each task) will periodically run as a background job. If you have many tests or monitors, you might want to set this value to match the number of running tests and monitors to prevent those jobs from queuing up and delaying the collecting of data. The current queue length can be seen as thescheduled_call_latencyparameter returned by thencc statuscommand.
/etc/apache2/sites-available/netrounds-ssl.confWSGIDaemonProcess netrounds ... processes: This configures how many HTTP requests to the Control Center GUI can be handled at the same time. The default is 10. If you have many or slow requests, you might need to increase this number. If no worker is available to start processing a request, a 504 Gateway Timeout response will be returned.