Troubleshooting 'Redis Connection Refused' and Service Failures

Understanding the 'Redis Connection Refused' Error

The Connection refused error is one of the most common issues developers face when interacting with Redis. At its core, this TCP/IP error implies that the client attempted to connect to a specific IP address and port (default 6379), but the operating system responded with an RST (reset) packet. This happens because there is no process listening on that port, or a firewall actively rejected the connection.

While the symptom is straightforward, the root cause can span a wide spectrum: from simple misconfigurations to catastrophic service crashes (e.g., OOM, segfaults), resource exhaustion (too many connections), or severe performance bottlenecks causing timeouts.

In this comprehensive guide, we will systematically diagnose and resolve the myriad reasons why Redis might be returning connection refused, crashing, or failing to start.

Scenario 1: The Redis Service is Not Starting or Has Crashed

If you receive a connection refused, your very first step should be verifying if the Redis process is actually running.

Diagnostics

Run systemctl status redis (or redis-server). If the service is listed as failed or inactive (dead), Redis has crashed or failed to start.

Next, examine the Redis crash log and system journal:

journalctl -u redis -n 100 --no-pager
tail -n 100 /var/log/redis/redis-server.log

Common Sub-causes:

1. Permission Denied: If the log shows Fatal error, can't open config file or Permission denied when accessing the RDB/AOF directory, the redis user lacks necessary filesystem permissions. Fix this by ensuring the data directory (usually /var/lib/redis) is owned by the redis user: chown -R redis:redis /var/lib/redis
2. Corrupt AOF/RDB: A sudden power loss can corrupt persistence files, preventing startup. Redis includes utilities to fix these. Run redis-check-aof --fix /var/lib/redis/appendonly.aof or redis-check-rdb to salvage the data.
3. Address Already in Use: If another process (or a zombie Redis instance) is bound to port 6379, the service will not start. Use netstat -tulpn | grep 6379 to find the offending PID and kill it.

Scenario 2: Network and Binding Issues

If the service is running perfectly but remote clients get Connection refused, the issue is network-related.

Diagnostics

Check what interfaces Redis is listening on: ss -tlnp | grep redis If it says 127.0.0.1:6379, it is bound to localhost. External connections will be actively refused.

Resolution

Open your redis.conf (usually in /etc/redis/redis.conf). Locate the bind directive. To allow external connections (from a private subnet, for instance), modify it:

# Change this:
bind 127.0.0.1 -::1

# To this (replace with your server's private IP):
bind 127.0.0.1 10.0.0.5

Warning: Never bind Redis to 0.0.0.0 (all interfaces) without implementing requirepass (password authentication) and firewall rules, as this exposes your datastore to the public internet.

Additionally, check your firewall (UFW, iptables, or cloud security groups) to ensure port 6379 is open for ingress from your application servers.

Scenario 3: Resource Exhaustion (Too Many Connections)

Redis has a finite limit on concurrent client connections. When this limit is reached, it will reject new connections, which might manifest as a connection refused or timeout at the application layer.

Diagnostics

You will see ERR max number of clients reached in your application logs or Redis logs. You can check the current limits and usage via redis-cli (if you can get a connection): INFO clients Look for connected_clients and maxclients.

Resolution

The default maxclients is 10,000. However, the actual limit is constrained by the operating system's open file descriptors limit (ulimit). If your OS limit is 1024, Redis will silently lower maxclients to 1024 minus overhead (usually ~992).

1. Increase OS file descriptor limits. Edit /etc/security/limits.conf:

   redis soft nofile 65536
   redis hard nofile 65536
   

2. Increase systemd limits if applicable. Run systemctl edit redis and add:

   [Service]
   LimitNOFILE=65536
   

3. Update redis.conf: maxclients 20000
4. Restart Redis.

Crucially, investigate why you have so many connections. Are your applications failing to close connections? Implement connection pooling in your client libraries.

Scenario 4: Redis OOM (Out of Memory) and the OOM Killer

Redis is an in-memory datastore. If it consumes all available system RAM, the Linux kernel's Out Of Memory (OOM) killer will forcibly terminate the Redis process to save the OS. This results in an immediate service crash and subsequent connection refusals.

Diagnostics

Check the system logs for OOM killer invocations: dmesg | grep -i oom-killer or grep -i 'killed process' /var/log/messages If you see redis-server in the output, you have a memory issue.

Resolution

1. Set maxmemory: Never run Redis in production without a maxmemory limit. It should be set to less than the physical RAM of the machine (leaving room for the OS and background saving operations). In redis.conf: maxmemory 4gb
2. Configure Eviction Policy: What should Redis do when it hits 4GB? The default is noeviction, meaning writes will start failing. If you use Redis as a cache, set an eviction policy like allkeys-lru or volatile-lru: maxmemory-policy allkeys-lru
3. Enable Overcommit Memory: Redis background saves (BGSAVE) fork the process. If Linux strict overcommit is enabled, the fork might fail or trigger OOM. Add vm.overcommit_memory = 1 to /etc/sysctl.conf and run sysctl -p.

Scenario 5: Performance Degradation (Redis Slow / High CPU)

Sometimes a connection isn't "refused" by the TCP stack, but it times out because Redis is unresponsive. Redis is single-threaded for command execution. If one command takes 5 seconds, all other clients are blocked for 5 seconds, causing massive connection backlogs and timeouts.

Diagnostics

1. Check CPU usage with top or htop. If the redis-server thread is pegged at 100%, it's blocking.
2. Use the Redis Slowlog to find the culprits:

   redis-cli slowlog get 10
   

   This shows the 10 slowest recent queries.

Resolution

• Ban KEYS *: The KEYS command is O(N) and blocks the entire server while scanning the keyspace. In production, always use SCAN instead, which iterates via a cursor and doesn't block.
• Analyze Large Keys: Commands operating on massive data structures (e.g., SMEMBERS on a set with 5 million items) will block. Find large keys using redis-cli --bigkeys.
• Disable THP (Transparent Huge Pages): THP can cause severe latency spikes in Redis during memory allocation. Disable it via your OS boot parameters or a systemd service.

By systematically checking the service state, network bindings, connection limits, memory constraints, and query performance, you can resolve almost any Redis connection refused or crash scenario.