ArgoCD 'connection refused' Error: Complete Troubleshooting Guide (2024)

Understanding the ArgoCD 'connection refused' Error

When you see dial tcp 127.0.0.1:443: connect: connection refused or failed to connect to server: connection refused in ArgoCD, it means either the argocd-server process is not listening, the Kubernetes Service is not routing correctly, or a firewall/network policy is dropping packets before they reach the pod. Unlike a timeout, a hard connection refused means the TCP handshake was actively rejected—the port is closed or the process is down.

ArgoCD exposes two primary interfaces: the gRPC API on port 8080 (used by the argocd CLI) and the HTTPS UI on port 443 (or 8080 in --insecure mode). Miscounting these ports, or running ArgoCD behind an ingress that terminates TLS incorrectly, is a frequent source of confusion.

Step 1: Verify All ArgoCD Pods Are Healthy

Begin with the most fundamental check—are the pods actually running?

kubectl get pods -n argocd -o wide
kubectl get events -n argocd --sort-by='.lastTimestamp' | tail -30

You should see these deployments in Running state with all containers ready:

• argocd-server
• argocd-repo-server
• argocd-application-controller
• argocd-redis
• argocd-dex-server (if SSO is enabled)

If any pod shows CrashLoopBackOff, ImagePullBackOff, or Pending, that is your primary issue.

Diagnosing CrashLoopBackOff

CrashLoopBackOff means the container starts and immediately exits. Kubernetes retries with exponential back-off. The error message you see in kubectl get pods is:

NAME                        READY   STATUS             RESTARTS   AGE
argocd-server-xxxx          0/1     CrashLoopBackOff   8          18m

Fetch the crash reason:

kubectl logs -n argocd deployment/argocd-server --previous
kubectl describe pod -n argocd -l app.kubernetes.io/name=argocd-server

Common crash causes and their log signatures:

• TLS secret missing: open /app/config/server/tls/tls.crt: no such file or directory
• Redis unreachable: Failed to connect to Redis: dial tcp: lookup argocd-redis
• Port conflict: bind: address already in use
• OOM: Container exits with code 137 (SIGKILL)

For TLS issues, regenerate the self-signed certificate:

kubectl delete secret argocd-server-tls -n argocd
kubectl rollout restart deployment argocd-server -n argocd

ArgoCD will auto-generate a new self-signed cert on startup.

Diagnosing ImagePullBackOff

This error means Kubernetes cannot pull the container image. The pod description shows:

Warning  Failed  2m  kubelet  Failed to pull image "quay.io/argoproj/argocd:v2.x.y": 
  rpc error: code = Unknown desc = failed to pull and unpack image: 
  failed to resolve reference "quay.io/argoproj/argocd:v2.x.y": 
  unexpected status code 401 Unauthorized

Diagnostic steps:

# Check if the image tag exists
docker manifest inspect quay.io/argoproj/argocd:v2.x.y

# Verify pull secret exists
kubectl get secret -n argocd | grep pull

# Check service account references
kubectl get serviceaccount argocd-server -n argocd -o yaml

If using a private registry or air-gapped environment, create and attach the pull secret:

kubectl create secret docker-registry argocd-pull-secret \
  --docker-server=your-registry.example.com \
  --docker-username=YOUR_USER \
  --docker-password=YOUR_PASS \
  -n argocd

kubectl patch serviceaccount argocd-server -n argocd \
  -p '{"imagePullSecrets": [{"name": "argocd-pull-secret"}]}'

Step 2: Verify the ArgoCD Service and Networking

Even when pods are running, the service may not route correctly.

kubectl get svc -n argocd
kubectl describe svc argocd-server -n argocd

For a LoadBalancer service that stays in <pending> state (common on bare-metal clusters), switch to NodePort or use kubectl port-forward to bypass the service layer entirely:

kubectl port-forward svc/argocd-server -n argocd 8080:443

Then test connectivity:

curl -k https://localhost:8080/healthz
# Expected: {"status":"ok"}

If /healthz responds but your ingress still gives connection refused, the problem is in your ingress controller or network policy, not ArgoCD itself.

Check Network Policies

kubectl get networkpolicy -n argocd

A restrictive NetworkPolicy that does not allow ingress on port 443 or 8080 to the argocd-server pod will produce a hard connection refused from the client's perspective. Add an explicit allow rule:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-argocd-server
  namespace: argocd
spec:
  podSelector:
    matchLabels:
      app.kubernetes.io/name: argocd-server
  ingress:
  - ports:
    - port: 8080
    - port: 8083

Step 3: Fix Permission Denied Errors

ArgoCD uses a service account with RBAC rules to interact with registered clusters. A permission denied error in ArgoCD logs typically looks like:

Failed to list *v1.Namespace: namespaces is forbidden: 
  User "system:serviceaccount:argocd:argocd-application-controller" 
  cannot list resource "namespaces" in API group "" at the cluster scope

This usually means the ClusterRoleBinding for argocd-application-controller is missing or its ClusterRole was narrowed. Reapply the official RBAC manifest:

kubectl apply -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

For external cluster registration, the cluster secret in the argocd namespace may have an expired bearer token:

# List registered clusters
argocd cluster list

# Re-register the cluster
argocd cluster add my-cluster-context --name my-cluster

# Verify the secret was updated
kubectl get secret -n argocd -l argocd.argoproj.io/secret-type=cluster

Step 4: Fix Timeout Errors

Timeout errors surface in two main ways:

1. CLI timeouts: rpc error: code = DeadlineExceeded desc = context deadline exceeded
2. Sync timeouts: Applications stuck in Progressing state beyond the configured timeout

For CLI timeouts, increase the gRPC timeout:

export ARGOCD_OPTS='--grpc-web --request-timeout 120s'
argocd app sync my-app

For repo-server timeouts (slow Git clones, large repos):

kubectl edit configmap argocd-cmd-params-cm -n argocd
# Add: repo-server.timeout.seconds: "180"

kubectl rollout restart deployment argocd-repo-server -n argocd

If the repo-server is OOMKilled under load, increase its resource limits:

kubectl patch deployment argocd-repo-server -n argocd --type=json \
  -p='[{"op":"replace","path":"/spec/template/spec/containers/0/resources",
       "value":{"requests":{"cpu":"500m","memory":"512Mi"},
                "limits":{"cpu":"2","memory":"2Gi"}}}]'

Step 5: Full Diagnostic Runbook

For systematic investigation, run this complete diagnostic sequence:

# 1. Overall pod health
kubectl get pods -n argocd

# 2. Recent events (shows OOM, failed mounts, pull errors)
kubectl get events -n argocd --sort-by='.lastTimestamp' | tail -50

# 3. argocd-server logs (last 200 lines, follow on crash)
kubectl logs -n argocd deployment/argocd-server --tail=200

# 4. repo-server logs
kubectl logs -n argocd deployment/argocd-repo-server --tail=100

# 5. application-controller logs
kubectl logs -n argocd statefulset/argocd-application-controller --tail=100

# 6. Service endpoints
kubectl get endpoints -n argocd argocd-server

# 7. Test internal connectivity from within cluster
kubectl run debug-pod --image=curlimages/curl --rm -it --restart=Never -n argocd \
  -- curl -k https://argocd-server.argocd.svc.cluster.local/healthz

# 8. Check ArgoCD version and config
kubectl get cm argocd-cmd-params-cm -n argocd -o yaml
kubectl get cm argocd-rbac-cm -n argocd -o yaml