GitLab CI Timeout: Fix Job Timeouts, Permission Denied & Pipelines Not Working
Resolve GitLab CI timeout errors and permission denied failures: increase job timeout in .gitlab-ci.yml, fix runner Docker permissions, and debug stuck pipeline
- GitLab CI jobs time out after 1 hour by default — increase per-job with 'timeout: 3h' in .gitlab-ci.yml or raise the project ceiling under Settings > CI/CD > General pipelines
- Permission denied on scripts means the executable bit is missing from git — fix permanently with 'git update-index --chmod=+x script.sh'; Docker socket permission denied requires adding gitlab-runner to the docker group
- Pipelines stuck in 'pending' almost always mean no runner is online, runner tags don't match the job, or the runner's max timeout is lower than the job needs — effective timeout is min(project_timeout, runner_max_timeout)
- Enable CI_DEBUG_TRACE: 'true' as a CI/CD variable to get verbose step-by-step runner output for any silent failure or hang
| Method | When to Use | Time to Implement | Risk |
|---|---|---|---|
| Add 'timeout: 3h' to job in .gitlab-ci.yml | Single job consistently exceeds the 1h limit | < 5 min | Low |
| Raise project timeout in Settings > CI/CD | All jobs need a higher ceiling | < 2 min | Low |
| usermod -aG docker gitlab-runner | Runner reports 'permission denied' on /var/run/docker.sock | 5–10 min | Medium — grants Docker daemon access |
| git update-index --chmod=+x | Script exits with '/bin/bash: ./script.sh: Permission denied' | < 5 min | Low |
| Re-register runner as correct user | Runner installed as root or wrong system user | 15–30 min | Low |
| Mount Docker socket in config.toml volumes | Docker builds inside jobs fail with socket errors | 10 min | Medium — shared socket |
| CI_DEBUG_TRACE: 'true' variable | Pipeline hangs silently with no visible error | < 5 min | Low — exposes env vars in logs |
Understanding GitLab CI Timeout, Permission Denied, and Pipeline Failures
GitLab CI/CD pipelines fail in predictable, fixable ways. This guide covers the three most common failure modes — job timeout, permission denied, and pipeline not triggering at all — with exact error messages, root cause analysis, and step-by-step remediation.
Part 1: GitLab CI Timeout
Exact error messages
When a job exceeds its configured timeout you will see one of the following in the job log:
ERROR: Job failed: execution took longer than 1h0m0s seconds
Job's activity exceeded the timeout
Running on runner-abc1234 via build-host-01
...
ERROR: Job failed (system failure): aborted: timeout
Root causes
Default 1-hour project timeout not adjusted. GitLab applies a 60-minute job timeout by default to every project. Large Rust or C++ compilations, slow integration test suites, multi-stage Docker builds with many layers, and database migration jobs frequently exceed this without any indication until they hit the wall.
Runner-level max timeout is lower than the job needs. Each registered runner has its own "Maximum job timeout" setting. The effective timeout for any job is min(project_timeout, runner_max_timeout). Raising the project setting to 3 hours does nothing if the runner is capped at 1 hour.
The job is hanging, not actually slow. Many apparent timeouts are really deadlocks: a test suite waiting on a network service that never responded, an interactive apt prompt blocking the script, or a parallel test runner that orphaned child processes.
Step 1: Determine whether the job is slow or hanging
Add coarse-grained timestamps to your script to pinpoint where time is spent:
build:
script:
- date && echo "[START] dependency install"
- npm ci --prefer-offline
- date && echo "[END] dependency install"
- date && echo "[START] build"
- npm run build
- date && echo "[END] build"
If the log stops mid-step with no further output until the timeout fires, the job is hanging. Enable CI_DEBUG_TRACE for verbose executor-level output:
variables:
CI_DEBUG_TRACE: "true"
Step 2: Increase the job-level timeout
Add timeout: directly under the job key in .gitlab-ci.yml. Accepted formats: 3h, 3h 30m, 210 minutes.
build-heavy:
stage: build
timeout: 3h
script:
- make all
This value cannot exceed the project-level ceiling or the runner's max timeout.
Step 3: Raise the project timeout ceiling
Go to Settings > CI/CD > General pipelines > Timeout and set a value that covers your worst-case job. For GitLab.com free tier, shared runners are hard-capped at 60 minutes. Paid tiers allow up to 3 hours on shared runners.
Step 4: Adjust the runner's max timeout
In Settings > CI/CD > Runners, expand the specific runner and update "Maximum job timeout". If you manage the runner yourself, you can also set this at registration time:
sudo gitlab-runner register \
--url https://gitlab.com \
--registration-token YOUR_TOKEN \
--executor docker \
--docker-image alpine:latest \
--maximum-timeout 10800
Step 5: Fix hanging jobs
For package managers that prompt interactively, force non-interactive mode:
before_script:
- export DEBIAN_FRONTEND=noninteractive
- apt-get update -y && apt-get install -y curl git
- npm ci --prefer-offline
For test suites, add explicit timeouts and a force-exit flag:
variables:
JEST_TIMEOUT: "30000"
script:
- npx jest --testTimeout=30000 --forceExit --bail
Part 2: GitLab CI Permission Denied
Exact error messages
/bin/bash: ./deploy.sh: Permission denied
dial unix /var/run/docker.sock: connect: permission denied
mkdir: cannot create directory '/cache': Permission denied
fatal: could not read Username for 'https://gitlab.com': No such device or address
Root causes
Script missing executable bit in git. Git tracks file permissions. If deploy.sh was added with git add without first running chmod +x, the file mode stored in git is 0644 (not executable). The runner checks out exactly what git has.
gitlab-runner user not in the docker group. On shell executor setups, the runner process runs as the gitlab-runner system user. By default this user is not in the docker group, so any command that opens /var/run/docker.sock fails immediately.
Artifact files owned by root. If a previous stage ran a Docker container that wrote files as uid 0, the next stage — running as gitlab-runner (typically uid 999 or similar) — cannot overwrite those files.
Missing deploy credentials for git push. CI jobs clone the repository read-only by default. Any script that calls git push without configuring credentials will fail with a permission error that looks like a network error.
Fix 1: Make scripts executable in git
This is the permanent fix — change the file mode stored in the git index:
git update-index --chmod=+x deploy.sh
git commit -m "fix: make deploy.sh executable"
git push
Verify the mode was stored correctly:
git ls-files -s deploy.sh
# Should show mode 100755, not 100644
Fix 2: Add gitlab-runner to the docker group
On the runner host:
sudo usermod -aG docker gitlab-runner
sudo systemctl restart gitlab-runner
# Verify membership
id gitlab-runner
# Expected output includes: groups=...,docker,...
# Spot-check access
sudo -u gitlab-runner docker ps
Fix 3: Mount the Docker socket in config.toml
Edit /etc/gitlab-runner/config.toml on the runner host:
[[runners]]
name = "my-runner"
executor = "docker"
[runners.docker]
image = "alpine:latest"
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
Restart after changes: sudo systemctl restart gitlab-runner
Fix 4: Correct artifact ownership for multi-stage pipelines
When a build stage uses Docker to produce artifacts, fix ownership before the artifacts block:
build:
script:
- docker run --rm -v "$PWD:/app" -w /app node:20 npm ci
- docker run --rm -v "$PWD:/app" -w /app node:20 npm run build
# Ensure next stage can read these files
- sudo chown -R "$(id -u):$(id -g)" dist/ node_modules/
artifacts:
paths:
- dist/
Fix 5: Git push from CI using a deploy token
Create a deploy token in Settings > Repository > Deploy tokens with write_repository scope, then store it as a masked CI/CD variable:
before_script:
- git remote set-url origin
"https://deploy-token:${CI_DEPLOY_PASSWORD}@${CI_SERVER_HOST}/${CI_PROJECT_PATH}.git"
- git config user.email "ci@example.com"
- git config user.name "GitLab CI"
Part 3: GitLab CI Not Working — Pipeline Stuck or Not Triggering
Symptom: Pipeline stays in 'Pending' indefinitely
This is almost always a runner availability problem. Check in order:
- Settings > CI/CD > Runners — is there at least one runner with a green online dot?
- If no online runners, SSH to the runner host and check the service:
sudo systemctl status gitlab-runner sudo gitlab-runner verify - Check for tag mismatch — if your job specifies
tags:, the runner must have matching tags AND "Run untagged jobs" must be disabled or the tags must match exactly (case-sensitive).
Symptom: Jobs never appear despite push
Validate your .gitlab-ci.yml against the GitLab lint API:
curl --silent \
--header "PRIVATE-TOKEN: $GITLAB_TOKEN" \
--header "Content-Type: application/json" \
--data "{\"content\": $(cat .gitlab-ci.yml | python3 -c 'import sys,json; print(json.dumps(sys.stdin.read()))')}" \
"https://gitlab.com/api/v4/ci/lint" | python3 -m json.tool
Also check for rules: or only: blocks that never evaluate to true for your branch. A common mistake:
# Wrong — pushes to 'master' never match
deploy:
rules:
- if: '$CI_COMMIT_BRANCH == "main"'
# Correct — uses the project's configured default branch
deploy:
rules:
- if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
Frequently Asked Questions
#!/usr/bin/env bash
# gitlab-ci-diagnose.sh — run on the GitLab Runner host
# Diagnoses timeout, permission denied, and pipeline-not-working issues
set -euo pipefail
HEADER() { echo ""; echo "=== $* ==="; }
HEADER "GitLab Runner Service Status"
sudo systemctl status gitlab-runner --no-pager -l
HEADER "Registered Runners"
sudo gitlab-runner list 2>&1
HEADER "Runner Connectivity (verify against GitLab)"
sudo gitlab-runner verify 2>&1
HEADER "Runner User Identity"
id gitlab-runner
HEADER "Docker Group Membership"
getent group docker || echo "docker group does not exist"
HEADER "Docker Socket Permissions"
ls -la /var/run/docker.sock
HEADER "Can gitlab-runner Access Docker?"
if sudo -u gitlab-runner docker info > /dev/null 2>&1; then
echo "OK: gitlab-runner can reach Docker daemon"
else
echo "FAIL: permission denied on Docker socket"
echo "FIX: sudo usermod -aG docker gitlab-runner && sudo systemctl restart gitlab-runner"
fi
HEADER "Runner Configuration"
sudo cat /etc/gitlab-runner/config.toml
HEADER "Effective Timeout Settings (from config.toml)"
sudo grep -E 'maximum_timeout|timeout' /etc/gitlab-runner/config.toml || echo "No explicit timeout set (runner default applies)"
HEADER "Disk Space (full disk causes silent failures)"
df -h
HEADER "Recent Runner Logs (last 60 lines)"
sudo journalctl -u gitlab-runner -n 60 --no-pager
HEADER "Git Executable Bits in .gitlab-ci.yml scripts (local repo)"
if [ -f .gitlab-ci.yml ]; then
# Extract script paths that start with ./ and check their git mode
grep -oP '\./.+\.sh' .gitlab-ci.yml | sort -u | while read -r script; do
mode=$(git ls-files -s "$script" 2>/dev/null | awk '{print $1}')
if [ "$mode" = "100644" ]; then
echo "WARN: $script is NOT executable in git (mode 100644)"
echo "FIX: git update-index --chmod=+x $script"
elif [ "$mode" = "100755" ]; then
echo "OK: $script is executable (mode 100755)"
else
echo "INFO: $script not tracked by git or not found"
fi
done
else
echo "No .gitlab-ci.yml found in current directory"
fi
HEADER "YAML Validation"
if command -v python3 &>/dev/null && python3 -c 'import yaml' 2>/dev/null; then
python3 - <<'PYEOF'
import yaml, sys
try:
with open('.gitlab-ci.yml') as f:
doc = yaml.safe_load(f)
jobs = [k for k,v in doc.items() if isinstance(v, dict) and 'script' in v]
print(f'OK: .gitlab-ci.yml is valid YAML with {len(jobs)} job(s): {", ".join(jobs)}')
except FileNotFoundError:
print('SKIP: .gitlab-ci.yml not found in current directory')
except yaml.YAMLError as e:
print(f'FAIL: YAML parse error: {e}')
sys.exit(1)
PYEOF
else
echo "Skipping — install python3-yaml: sudo apt-get install -y python3-yaml"
fi
echo ""
echo "Diagnostic complete."Error Medic Editorial
The Error Medic Editorial team is composed of senior DevOps and SRE engineers with hands-on experience managing GitLab CI/CD at scale across cloud-native, on-premises, and hybrid environments. Our guides are written from production incident retrospectives and peer-reviewed for technical accuracy before publication.
Sources
- https://docs.gitlab.com/ee/ci/runners/configure_runners.html#set-maximum-job-timeout-for-a-runner
- https://docs.gitlab.com/ee/ci/yaml/index.html#timeout
- https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-socket-binding
- https://docs.gitlab.com/ee/ci/troubleshooting.html
- https://stackoverflow.com/questions/44094754/gitlab-ci-runner-permission-denied-on-docker-sock
- https://gitlab.com/gitlab-org/gitlab-runner/-/issues/1986