How to Fix 'Memcached Connection Refused' and Server Crashes

Understanding the Error

The memcached connection refused error is a TCP/IP level rejection. When your application (whether it's written in PHP, Python, Node.js, or Ruby) attempts to open a socket connection to the Memcached server on port 11211, the operating system actively rejects the connection. This manifests in application logs as errors like Memcached::connect(): Server connection failure or Error: connect ECONNREFUSED 127.0.0.1:11211.

This rejection happens for one of three primary reasons:

1. The Daemon is Dead: The Memcached process has crashed or was never started. This is often linked to the memcached not working or memcached crash symptoms.
2. Wrong Bind Address: The Memcached process is running, but it is bound to 127.0.0.1 (localhost), while your application is trying to connect from a remote server.
3. Firewall Blocking: A firewall (like iptables or ufw) is dropping or rejecting the packets before they reach the Memcached process.

Another critical scenario is when Memcached is running but unresponsive or dropping connections, often described as memcached slow or memcached out of memory.

Step 1: Diagnose the Root Cause

Before changing configurations, you must identify why the connection is being refused.

1. Check Service Status The first step is to see if the Memcached process is actually running. Run systemctl status memcached. If the service shows as failed or inactive, you have found the immediate reason for the connection refusal.

2. Investigate Crashes (The OOM Killer) If the service was running but stopped unexpectedly, check the system logs for the Linux Out of Memory (OOM) killer. Memcached is designed to use a specific amount of memory. However, if the overall system runs out of RAM, the kernel will kill processes to survive. Run dmesg -T | grep -i oom or grep -i 'out of memory' /var/log/syslog. If you see Memcached listed, you are dealing with a memcached out of memory crash.

3. Verify Port Binding If the service is running, verify which IP addresses it is listening on. Run ss -tulpn | grep 11211. If the output shows 127.0.0.1:11211, Memcached will only accept connections from the local machine. If your app is on another server, the connection will be refused.

4. Check Connection Limits If Memcached is slow or intermittently refusing connections, you might be hitting the maximum connection limit. Connect to Memcached via telnet (telnet localhost 11211), type stats, and look at the curr_connections and listen_disabled_num metrics. If listen_disabled_num is greater than 0, Memcached has stopped accepting connections because the limit was reached.

Step 2: Implement the Fix

Fixing Bind Address Issues To allow remote connections, you must change the bind address in the Memcached configuration file, typically located at /etc/memcached.conf (Debian/Ubuntu) or /etc/sysconfig/memcached (RHEL/CentOS). Find the line -l 127.0.0.1 and change it to the server's private network IP, or -l 0.0.0.0 to listen on all interfaces (ensure you have a firewall in place if you do this!).

# /etc/memcached.conf
-l 10.0.0.5

Restart the service: systemctl restart memcached.

Fixing Out of Memory Crashes If Memcached is crashing due to OOM, you have two options:

1. Decrease the memory allocated to Memcached (the -m flag) so it leaves enough room for the OS and other processes.
2. Add more physical RAM to the server.

In /etc/memcached.conf, adjust the -m value (in Megabytes):

# Allocate 1GB to Memcached
-m 1024

Note: Memcached's -m flag only limits the memory used for item storage. The process itself will use slightly more memory for connections and management structures. Always leave a buffer for the OS.

Fixing Connection Limits and Slow Performance If you are hitting connection limits, increase the -c flag (default is usually 1024).

# /etc/memcached.conf
# Increase max simultaneous connections
-c 4096

If memcached slow is your primary symptom, check the evictions metric in the stats output. High evictions mean Memcached is out of space and is deleting old items to make room for new ones. You need to increase the -m memory limit.

Step 3: Validation and Hardening

After applying fixes, validate the setup:

1. Local test: echo "stats" | nc localhost 11211
2. Remote test: From your application server, run nc -zv <memcached_ip> 11211 to ensure the port is open and reachable.

Security Warning: Never expose Memcached directly to the public internet (0.0.0.0 without a firewall). It has no built-in authentication by default and can be used in UDP amplification DDoS attacks or have its data easily stolen. Always use firewall rules (like ufw allow from <app_server_ip> to any port 11211) or bind exclusively to a private VPC/network interface.