502 Bad Gateway Error: Complete Troubleshooting Guide

Understanding 502 Bad Gateway Errors

A 502 Bad Gateway error occurs when a server acting as a gateway or proxy receives an invalid response from an upstream server. This is fundamentally different from other HTTP errors:

• 403 Forbidden: Client lacks permission to access resource
• 404 Not Found: Resource doesn't exist at specified URL
• 503 Service Unavailable: Server temporarily unable to handle requests
• 504 Gateway Timeout: Upstream server fails to respond within timeout period

The 502 error specifically indicates communication failure between proxy and backend server, making it a critical infrastructure issue requiring immediate attention.

Common 502 Bad Gateway Scenarios

Nginx Reverse Proxy Issues Nginx serving as reverse proxy encounters backend server problems:

2024/01/15 10:30:45 [error] 1234#0: *567 connect() failed (111: Connection refused) while connecting to upstream
2024/01/15 10:30:45 [error] 1234#0: *567 upstream prematurely closed connection while reading response header from upstream

Application Server Crashes PHP-FPM, Node.js, or Python applications crash while nginx attempts connection:

502 Bad Gateway
nginx/1.23.1

Load Balancer Failures AWS Application Load Balancer or Azure Application Gateway returns 502 when backend instances fail health checks.

Docker Container Issues Docker containers exit unexpectedly while nginx proxy attempts connection:

502 Bad Gateway
openresty/1.19.9.1

Step 1: Initial Diagnosis

Check Server Logs Immediately Start with web server error logs to identify root cause:

# Nginx error logs
sudo tail -f /var/log/nginx/error.log

# Apache error logs
sudo tail -f /var/log/apache2/error.log

# System logs for service failures
sudo journalctl -u nginx -f
sudo journalctl -u php7.4-fpm -f

Verify Upstream Service Status Check if backend services are running:

# Check process status
sudo systemctl status nginx
sudo systemctl status php7.4-fpm
sudo systemctl status mysql

# Check port availability
sudo netstat -tlnp | grep :9000  # PHP-FPM
sudo netstat -tlnp | grep :3000  # Node.js
sudo ss -tlnp | grep :8080       # Alternative check

Test Direct Backend Connection Bypass proxy to test backend directly:

# Test PHP-FPM socket
sudo -u www-data php-fpm -t

# Test application port directly
curl -v http://localhost:3000/health
curl -v http://127.0.0.1:8080/status

# Check database connectivity
mysqladmin -u root -p ping

Step 2: Nginx-Specific Troubleshooting

Common Nginx 502 Causes and Fixes

1. PHP-FPM Not Running

# Restart PHP-FPM
sudo systemctl restart php7.4-fpm

# Check PHP-FPM configuration
sudo nginx -t
sudo php-fpm7.4 -t

2. Incorrect Upstream Configuration

# Fix common nginx upstream config
upstream backend {
    server 127.0.0.1:9000;  # Correct PHP-FPM
    # server unix:/var/run/php/php7.4-fpm.sock;  # Alternative socket
}

server {
    location ~ \.php$ {
        fastcgi_pass backend;
        fastcgi_index index.php;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    }
}

3. Timeout Issues

# Increase timeout values in nginx.conf
http {
    proxy_connect_timeout 60s;
    proxy_send_timeout 60s;
    proxy_read_timeout 60s;
    fastcgi_connect_timeout 60s;
    fastcgi_send_timeout 60s;
    fastcgi_read_timeout 60s;
}

Step 3: Application Server Troubleshooting

PHP-FPM Issues

# Check PHP-FPM pool configuration
sudo nano /etc/php/7.4/fpm/pool.d/www.conf

# Verify socket permissions
ls -la /var/run/php/php7.4-fpm.sock

# Monitor PHP-FPM processes
sudo tail -f /var/log/php7.4-fpm.log

Node.js Application Issues

# Check Node.js process
pm2 status
pm2 logs

# Restart Node.js application
pm2 restart all

# Check application logs
tail -f /var/log/node-app.log

Python/Django Issues

# Check Gunicorn status
sudo systemctl status gunicorn

# Restart Gunicorn
sudo systemctl restart gunicorn

# Check Gunicorn configuration
sudo nano /etc/systemd/system/gunicorn.service

Step 4: Load Balancer and Cloud Platform Fixes

AWS Application Load Balancer 502s

# Check target group health
aws elbv2 describe-target-health --target-group-arn arn:aws:elasticloadbalancing:...

# Verify security group rules
aws ec2 describe-security-groups --group-ids sg-12345678

# Check CloudWatch logs
aws logs filter-log-events --log-group-name /aws/applicationloadbalancer/app/my-loadbalancer

Azure Application Gateway Issues

# Check backend health via Azure CLI
az network application-gateway show-backend-health --name myAppGateway --resource-group myRG

# Verify NSG rules
az network nsg show --name myNSG --resource-group myRG

Step 5: SSL/TLS and ERR_SSL_PROTOCOL_ERROR Resolution

Certificate Issues Causing 502s

# Check SSL certificate validity
openssl x509 -in /etc/ssl/certs/domain.crt -text -noout

# Test SSL connection
openssl s_client -connect domain.com:443

# Verify certificate chain
curl -vI https://domain.com

Fix SSL Configuration in Nginx

server {
    listen 443 ssl http2;
    ssl_certificate /etc/ssl/certs/domain.crt;
    ssl_certificate_key /etc/ssl/private/domain.key;
    ssl_protocols TLSv1.2 TLSv1.3;
    ssl_ciphers ECDHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384;
    
    location / {
        proxy_pass http://backend;
        proxy_ssl_verify off;  # If backend uses self-signed cert
    }
}

Step 6: Docker and Container Issues

Docker Container 502 Debugging

# Check container status
docker ps -a
docker logs container_name

# Restart containers
docker-compose down
docker-compose up -d

# Check container networking
docker network ls
docker network inspect bridge

# Test inter-container connectivity
docker exec nginx_container ping app_container

Step 7: Performance and Resource Issues

Resource Exhaustion Causing 502s

# Check system resources
top
htop
free -h
df -h

# Check process limits
sudo -u www-data ulimit -a

# Monitor connection counts
netstat -an | grep :80 | wc -l
ss -s

Optimize Resource Settings

# Nginx worker optimization
worker_processes auto;
worker_connections 1024;
worker_rlimit_nofile 2048;

events {
    use epoll;
    multi_accept on;
}

Step 8: Monitoring and Prevention

Set Up Proactive Monitoring

# Create health check script
#!/bin/bash
STATUS=$(curl -o /dev/null -s -w "%{http_code}" http://localhost/health)
if [ $STATUS -ne 200 ]; then
    echo "Server returned $STATUS" | mail -s "Server Alert" admin@domain.com
    systemctl restart nginx
fi

# Add to crontab
*/5 * * * * /usr/local/bin/health_check.sh

Implement Circuit Breaker Pattern

# Nginx upstream with health checks
upstream backend {
    server 127.0.0.1:8001 max_fails=3 fail_timeout=30s;
    server 127.0.0.1:8002 max_fails=3 fail_timeout=30s backup;
}