What's Actually Happening
Nginx worker processes consume excessive CPU resources, causing server slowdown, timeouts, and affecting other services. A single worker may use 90-100% CPU or all workers combined consume most CPU capacity.
The Error You'll See
High CPU usage:
```bash $ top -p $(pgrep nginx)
PID USER CPU MEM COMMAND 1234 nginx 95% 2M nginx: worker process 1235 nginx 85% 2M nginx: worker process 1236 nginx 90% 2M nginx: worker process
$ ps aux | grep nginx
nginx 1234 95.0 0.1 nginx: worker process nginx 1235 85.0 0.1 nginx: worker process ```
System load high:
```bash $ uptime
load average: 15.0, 12.0, 10.00 # Very high for 4 CPU cores ```
Request timeouts:
```bash $ curl -w "%{time_total}" http://localhost/
10.5 # Slow response time ```
Why This Happens
- 1.Too many connections - Worker processes handling excessive requests
- 2.SSL/TLS overhead - High SSL handshake CPU cost
- 3.Large buffers - Excessive memory allocation/copying
- 4.Regex in configs - Complex rewrite rules
- 5.Too few workers - Single worker overloaded
- 6.Blocking operations - Slow upstream responses
Step 1: Identify High CPU Worker
```bash # Check CPU usage by worker processes top -H -p $(pgrep -d',' nginx)
# Or with ps ps -L aux | grep nginx
# Find specific worker with high CPU pidstat -p $(pgrep -d',' nginx) 1 5
# Check worker process count ps aux | grep "nginx: worker" | wc -l
# Compare to nginx config grep worker_processes /etc/nginx/nginx.conf
# Check worker connections limit grep worker_connections /etc/nginx/nginx.conf
# Check worker CPU affinity (if set) grep worker_cpu_affinity /etc/nginx/nginx.conf ```
Step 2: Check Current Connections
```bash # Check active connections nginx_status() { curl -s http://localhost/nginx_status }
# Or via stub_status module: curl http://localhost/nginx_status
Active connections: 5000 server accepts handled requests 10000 10000 10000 Reading: 100 Writing: 4000 Waiting: 900
# Check worker connections per worker ps aux | grep "nginx: worker" | wc -l # Divide active connections by workers
# Check connection state ss -tnp | grep nginx
# Check socket backlog ss -ltnp | grep nginx
# Check if hitting connection limits grep worker_connections /etc/nginx/nginx.conf # Default: 1024 ```
Step 3: Optimize Worker Process Count
```bash # Check current worker count grep worker_processes /etc/nginx/nginx.conf
# Auto-detect CPU cores (recommended) worker_processes auto;
# Or set to number of CPU cores worker_processes 4; # For 4 CPU cores
# Check CPU cores available nproc lscpu
# Reload nginx nginx -s reload
# Or restart systemctl restart nginx
# Verify worker count ps aux | grep "nginx: worker process" | grep -v grep | wc -l ```
Step 4: Optimize Worker Connections
```bash # Check current limit grep worker_connections /etc/nginx/nginx.conf
# Default is 1024, increase for high traffic events { worker_connections 4096; # Increase based on traffic multi_accept on; # Accept multiple connections at once use epoll; # Linux efficient event model }
# Maximum connections = worker_processes * worker_connections # For 4 workers with 4096 connections = 16384 total
# Check file descriptor limit ulimit -n
# Increase system limit # In /etc/security/limits.conf: nginx soft nofile 65536 nginx hard nofile 65536
# Or in nginx systemd service: # /etc/systemd/system/nginx.service [Service] LimitNOFILE=65536
# Reload systemd systemctl daemon-reload systemctl restart nginx ```
Step 5: Reduce SSL CPU Overhead
```bash # SSL/TLS requires significant CPU for handshakes
# Check SSL configuration grep -r ssl_ /etc/nginx/
# Use session caching to reduce handshake CPU ssl_session_cache shared:SSL:50m; ssl_session_timeout 1d; ssl_session_tickets off; # Or on for session tickets
# Use efficient cipher suites ssl_ciphers 'ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384'; ssl_prefer_server_ciphers on;
# Enable OCSP stapling (reduces client verification CPU) ssl_stapling on; ssl_stapling_verify on;
# Check CPU usage with SSL vs non-SSL # Test handshake performance openssl s_time -connect localhost:443
# For heavy SSL traffic, consider: # - SSL offloading to load balancer # - Hardware SSL accelerators # - Fewer cipher suite negotiations ```
Step 6: Optimize Buffer Sizes
```bash # Check buffer configuration grep -E "buffer|buffer_size" /etc/nginx/nginx.conf
# Large buffers cause memory allocation overhead
# Optimize client body buffer client_body_buffer_size 16k; # Default 8k or 16k
# Optimize output buffers output_buffers 2 32k; # Number and size
# Optimize fastcgi buffers (for PHP) fastcgi_buffer_size 32k; fastcgi_buffers 8 16k;
# Optimize proxy buffers proxy_buffer_size 32k; proxy_buffers 8 16k;
# Avoid too large buffers - balance between disk I/O and memory
# Check memory usage nginx -V 2>&1 | grep -i malloc ps aux | grep nginx | awk '{print $6}' # RSS in KB ```
Step 7: Optimize Regex and Rewrite Rules
```bash # Check rewrite rules grep -r rewrite /etc/nginx/
# Regex is CPU-intensive
# Example inefficient rule: rewrite ^/(.*)/([0-9]+)/([a-z]+)$ /page.php?cat=$1&id=$2&name=$3 last;
# Optimize: # 1. Use exact matches where possible location = /specific-path { # Exact match, no regex }
# 2. Use simpler patterns rewrite ^/([0-9]+)$ /page?id=$1 last;
# 3. Avoid regex for static files location ~* \.(jpg|png|css|js)$ { # Cache these, no complex regex }
# 4. Use map for complex mappings map $uri $new_uri { /old-path /new-path; /another /redirect; }
rewrite ^ $new_uri permanent;
# Benchmark regex performance # nginx -t to test configuration ```
Step 8: Optimize Keepalive Connections
```bash # Keepalive reduces CPU by reusing connections
# Check keepalive settings grep -E "keepalive|keepalive_timeout" /etc/nginx/nginx.conf
# Optimize client keepalive keepalive_timeout 65; # Default 75, reduce if many idle connections keepalive_requests 100; # Requests per connection
# Optimize upstream keepalive upstream backend { server backend1:8080; keepalive 32; # Keep 32 connections open to upstream }
proxy_http_version 1.1; proxy_set_header Connection "";
# This reduces upstream connection CPU overhead
# Check keepalive effectiveness curl -I http://localhost/ # Look for "Connection: keep-alive" header ```
Step 9: Profile Nginx CPU Usage
```bash # Use perf to profile CPU usage perf top -p $(pgrep nginx) # Live view of hot functions
# Record CPU usage perf record -p $(pgrep nginx) -g -- sleep 30
# Analyze perf report
# Common hotspots: # - SSL handshake (ngx_ssl_handshake) # - epoll_wait (event loop) # - buffer operations (ngx_http_write_filter)
# Use strace for syscall analysis strace -p $(pgrep nginx | head -1) -c
# Common syscalls: # - epoll_wait (event polling) # - read/write (I/O) # - accept (new connections)
# Benchmark configuration changes ab -n 10000 -c 100 http://localhost/ wrk -t 4 -c 100 -d 30s http://localhost/ ```
Step 10: Monitor Nginx Performance
```bash # Enable stub_status for monitoring # In nginx.conf: location /nginx_status { stub_status on; allow 127.0.0.1; deny all; }
# Monitor script cat << 'EOF' > monitor_nginx.sh #!/bin/bash echo "=== Worker CPU Usage ===" pidstat -p $(pgrep -d',' nginx) 1 1 | tail -n +4
echo "" echo "=== Connection Stats ===" curl -s http://localhost/nginx_status
echo "" echo "=== Worker Process Count ===" ps aux | grep "nginx: worker process" | grep -v grep | wc -l
echo "" echo "=== System Load ===" uptime EOF
chmod +x monitor_nginx.sh
# Prometheus/nginx-prometheus-exporter # Export nginx metrics for monitoring
# Monitor with ngxtop pip install ngxtop ngxtop -l /var/log/nginx/access.log ```
Nginx CPU Optimization Checklist
| Check | Command | Expected |
|---|---|---|
| Worker count | ps aux | Matches CPU cores |
| Worker CPU | top -H | < 70% per worker |
| Connections | nginx_status | Below limit |
| SSL sessions | nginx -V | Session caching enabled |
| Keepalive | curl -I | Keep-alive header |
| Buffer size | grep buffer | Reasonable sizes |
Verify the Fix
```bash # After optimizing nginx configuration
# 1. Reload nginx nginx -s reload
# 2. Check worker CPU usage top -H -p $(pgrep nginx) // Should show lower CPU percentages
# 3. Monitor connections curl http://localhost/nginx_status // Active connections reasonable
# 4. Test response time curl -w "%{time_total}\n" http://localhost/ // Should be faster
# 5. Benchmark throughput wrk -t 4 -c 100 -d 30s http://localhost/ // Should handle requests efficiently
# 6. Check system load uptime // Should be lower ```
Related Issues
- [Fix Nginx High Memory Usage](/articles/fix-nginx-high-memory-usage)
- [Fix Nginx Connection Timeout](/articles/fix-nginx-connection-timeout)
- [Fix Nginx 502 Bad Gateway](/articles/fix-nginx-502-bad-gateway)