Terraform State Locked: Fix 'Error locking state' and Access Denied Issues

Understanding the Terraform State Lock Error

Terraform uses a locking mechanism to protect state files from simultaneous writes, which would corrupt shared infrastructure state. When a terraform apply or terraform plan starts, it attempts to acquire a lock on the backend. If the lock cannot be acquired — or if a previous process crashed without releasing it — you will see one of the following errors:

Error: Error locking state: Error acquiring the state lock: ConditionalCheckFailedException: The conditional request failed
Lock Info:
  ID:        a1b2c3d4-e5f6-7890-abcd-ef1234567890
  Path:      s3://my-bucket/terraform.tfstate
  Operation: OperationTypeApply
  Who:       user@hostname
  Version:   1.6.0
  Created:   2024-11-12 09:34:21.123456789 +0000 UTC
  Info:

Or for access-related failures:

Error: Failed to get existing workspaces: S3 bucket does not exist.

Error: error using credentials to get account ID: operation error STS: GetCallerIdentity,
https response error StatusCode: 403, RequestID: ..., api error AccessDenied: Access denied

Or for permission denied on the lock table:

Error: Error acquiring the state lock: 2 errors occurred:
  * ResourceNotFoundException: Requested resource not found
  * AccessDeniedException: User: arn:aws:iam::123456789012:user/deploy is not authorized
    to perform: dynamodb:PutItem on resource: arn:aws:dynamodb:us-east-1:...

Step 1: Identify the Error Category

Before acting, categorize the failure:

Category A — Stale Lock (process crashed or CI job killed) The lock ID is printed in the error output. No active Terraform process is running. This is the most common scenario after a CI pipeline timeout or a laptop lid-close during apply.

Category B — Access Denied / Permission Denied The error mentions AccessDeniedException, 403, or permission denied. The process cannot even create or check the lock. This is an IAM or credential issue.

Category C — Timeout / Network The error says RequestError: send request failed or context deadline exceeded. The process can reach the backend but gets no timely response. This is infrastructure-level.

Category D — Corrupt State or Missing Lock Table The error mentions ResourceNotFoundException or the state file is malformed. The DynamoDB table or GCS/Azure equivalent does not exist or the state JSON is invalid.

Step 2: Diagnose the Root Cause

For Stale Locks (Category A)

First, confirm no other process holds the lock legitimately:

# Check for running Terraform processes on CI/CD
# In GitHub Actions: check active workflow runs
gh run list --workflow=terraform.yml --status=in_progress

# On AWS, check the DynamoDB lock table directly
aws dynamodb get-item \
  --table-name terraform-state-locks \
  --key '{"LockID": {"S": "my-bucket/terraform.tfstate"}}' \
  --region us-east-1

The response will show WHO created the lock and when. If the timestamp is older than your longest possible apply window (e.g., > 60 minutes) or the WHO field points to a hostname that no longer exists, it is safe to force-unlock.

For Access Denied (Category B)

# Verify current credentials
aws sts get-caller-identity

# Test specific permissions needed for S3 backend + DynamoDB locking
aws s3 ls s3://your-terraform-state-bucket/
aws dynamodb describe-table --table-name terraform-state-locks

# Check what policy is attached
aws iam get-user --user-name deploy-user
aws iam list-attached-user-policies --user-name deploy-user
aws iam list-user-policies --user-name deploy-user

The minimum required IAM permissions for an S3+DynamoDB backend are:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:ListBucket"
      ],
      "Resource": [
        "arn:aws:s3:::your-terraform-state-bucket",
        "arn:aws:s3:::your-terraform-state-bucket/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:GetItem",
        "dynamodb:PutItem",
        "dynamodb:DeleteItem"
      ],
      "Resource": "arn:aws:dynamodb:us-east-1:123456789012:table/terraform-state-locks"
    }
  ]
}

For Timeouts (Category C)

# Check DynamoDB table metrics in CloudWatch
aws cloudwatch get-metric-statistics \
  --namespace AWS/DynamoDB \
  --metric-name ThrottledRequests \
  --dimensions Name=TableName,Value=terraform-state-locks \
  --start-time $(date -u -d '1 hour ago' +%Y-%m-%dT%H:%M:%SZ) \
  --end-time $(date -u +%Y-%m-%dT%H:%M:%SZ) \
  --period 300 \
  --statistics Sum

# Test network connectivity to S3 endpoint
curl -v https://s3.us-east-1.amazonaws.com/ 2>&1 | head -20

Step 3: Apply the Fix

Fix A — Force-Unlock a Stale Lock

# Use the Lock ID from the error output
terraform force-unlock a1b2c3d4-e5f6-7890-abcd-ef1234567890

Terraform will prompt for confirmation. After unlocking, immediately run:

terraform plan

This confirms state integrity before proceeding.

Fix B — Manual DynamoDB Delete (when force-unlock fails)

# Replace the LockID value with your exact state path
aws dynamodb delete-item \
  --table-name terraform-state-locks \
  --key '{"LockID": {"S": "your-bucket/path/to/terraform.tfstate"}}' \
  --region us-east-1

Warning: Double-check the LockID string exactly matches — deleting the wrong item will have no effect but deleting the wrong table item in a multi-workspace setup can corrupt another team's lock.

Fix C — Resolve Access Denied

Attach the corrected policy:

# Create the policy document
cat > terraform-backend-policy.json << 'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {"Effect": "Allow", "Action": ["s3:GetObject","s3:PutObject","s3:DeleteObject","s3:ListBucket"], "Resource": ["arn:aws:s3:::YOUR_BUCKET","arn:aws:s3:::YOUR_BUCKET/*"]},
    {"Effect": "Allow", "Action": ["dynamodb:GetItem","dynamodb:PutItem","dynamodb:DeleteItem"], "Resource": "arn:aws:dynamodb:REGION:ACCOUNT_ID:table/YOUR_TABLE"}
  ]
}
EOF

aws iam put-user-policy \
  --user-name deploy-user \
  --policy-name TerraformBackendAccess \
  --policy-document file://terraform-backend-policy.json

Fix D — Create Missing DynamoDB Table

If the lock table was accidentally deleted:

aws dynamodb create-table \
  --table-name terraform-state-locks \
  --attribute-definitions AttributeName=LockID,AttributeType=S \
  --key-schema AttributeName=LockID,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST \
  --region us-east-1

Then re-run terraform init to re-register the backend.

Step 4: Prevent Recurrence

1. Set pipeline timeouts: In CI/CD, always set a hard timeout and ensure the cleanup step runs terraform force-unlock with the lock ID captured at apply-start.
2. Use workspace isolation: Separate workspaces (or separate state files) per environment prevent a dev lock from blocking production.
3. Enable DynamoDB auto-scaling: Prevents throttling-induced lock timeouts at scale.
4. Tag lock table with cost allocation tags: Makes it easier to identify ownership and avoid accidental deletion.
5. Monitor for stale locks: Set a CloudWatch alarm on DynamoDB items older than N minutes using a Lambda or EventBridge rule.