Performance tuning NGINX and PHP-fpm is a black art. Every case is different and there is no strict right or wrong way to do it. This is a description of the steps we made in our specific set of circumstances. We expect there are many more improvements we can make.
The Scenario
We were recently tasked with hosting and configuring a very busy Xenforo forum for one of our customers. The infrastructure is hosted in AWS.
After initial testing, we were happy with our instance sizing and auto-scaling configurations.
- EC2 instances: c5.large (2 vCPU, 4GB RAM)
- RDS: MariaDB 10.3.13, r4.large (2 vCPU, 15.25 GB RAM)
- Auto-scaled at 600 requests per instance.
At an OS and middleware level we were using:
- Ubuntu 18.04
- Xenforo 2.1
- Nginx 1.14
- PHP 7.2
The Challenge
The configuration was adequate and mostly default, but we wanted to fine tune the middleware and squeeze more out of the infrastructure where possible. Even during peak load there was still a lot of available RAM and CPU. We wanted to utilize this.
The areas we focused on were NGINX and PHP-fpm.
PHP-fpm Tuning
Changes from the default PHP-fpm configuration.
1/ We tuned PHP-fpm to better match our server spec.
The default is to use dynamic settings which we kept.
The recommendation for pm.max_children, is available memory divided by average memory of the child processes. Our php-fpm processes are around 50MB so we chose 75 pm.max.children.
We tuned it to start at 50 processes to avoid scaling of processes (not EC2 instances) too regularly.
pm.max_children = 75 pm.start_servers = 50 pm.min_spare_servers = 50 pm.max_spare_servers = 80 pm.max_requests = 50
We have since noticed that the number of child processes does not go above the initial 50 mark, so we clearly still have some spare capacity.
2/ Normally all requests are channeled to a single PHP-fpm instance. We chose to load balance requests across two PHP-fpm listeners to avoid possible bottlenecks.
We created a second .sock2 file, /run/php/php7.2-fpm.sock2, which does not exist by default.
We added the upstream parameter and referenced the .sock files in the site’s Nginx config:
upstream backend {
server unix:/var/run/php/php7.2-fpm.sock1 weight=100 max_fails=5 fail_timeout=5;
server unix:/var/run/php/php7.2-fpm.sock2 weight=100 max_fails=5 fail_timeout=5;We then changed the existing fastcgi_pass value to reference backend.
fastcgi_pass backend;
Next we added a second pool to /etc/php/7.2/fpm/pool.d replacing [www] with:
[www1] listen = /run/php/php7.2-fpm.sock1
and
[www2] listen = /run/php/php7.2-fpm.sock2
NGINX Tuning
We added:
Worker Connections to increase it from the default value.
worker_connections 5000;
Client buffer settings:
client_max_body_size 20M; client_body_buffer_size 1m; # was 10K client_header_buffer_size 1k; large_client_header_buffers 4 16k;
Decreased the keepalive_timeout to free up resources more quickly:
keepalive_timeout 2 2; # was 20
Added proxy settings, required for keepalive_timeout to upstream servers:
proxy_http_version 1.1; proxy_set_header Connection "";
Added gzip configuration:
gzip on; gzip_disable "msie6"; gzip_vary on; gzip_proxied any; gzip_comp_level 6; gzip_min_length 1100; gzip_buffers 16 8k; gzip_http_version 1.1; gzip_types text/plain application/javascript application/json application/x-javascript text/xml text/css;
Added open file cache settings:
open_file_cache max=2000 inactive=20s; open_file_cache_valid 60s; open_file_cache_min_uses 5; open_file_cache_errors off;
Under the fastcgi settings we removed the send and read timeouts, as it masks underlying issues. We reduced the connect_timeout and re-balanced the fastcgi_buffers.
include fastcgi_params; fastcgi_intercept_errors on; fastcgi_ignore_client_abort off; fastcgi_connect_timeout 3s; # was 60 # fastcgi_send_timeout 180; # fastcgi_read_timeout 180; fastcgi_buffer_size 128k; fastcgi_buffers 128 16k; # was 4 256k fastcgi_busy_buffers_size 256k; fastcgi_temp_file_write_size 256k; reset_timedout_connection on;
Outcome
This process reduced the amount of wasted available RAM and CPU.
- We are splitting load across both CPU cores by load balancing two PHP-fpm instances.
- We have more child PHP-fpm processes standing by to take new connections.
- We have raised the number of connections each NGINX worker process can handle simultaneously.
- We are closing connections much quicker, freeing up resources for new requests.
- We have removed or shortened the fastcgi timeout configuration so we can spot problems in the log more easily.
- We have added gzip and client caching optimizations.
References
We found the following blog posts and official guides invaluable for optimization recommendations and for giving a general understanding of PHP-fpm and NGINX configuration.