Troubleshooting GitHub API Rate Limit (403, 429) and Authentication (401) Errors

Understanding GitHub API Errors

When integrating with GitHub, whether for CI/CD pipelines, automation scripts, or custom dashboards, you will inevitably encounter API errors. The most common issues revolve around rate limiting and authentication. Understanding the difference between 401 Unauthorized, 403 Forbidden, 429 Too Many Requests, and 502 Bad Gateway is crucial for building resilient integrations.

The Rate Limit Thresholds (403 and 429)

GitHub imposes strict limits on how many API requests you can make within a one-hour window. This ensures platform stability for all users.

• Unauthenticated Requests: Limited to 60 requests per hour per IP address. This is extremely easy to hit, even with simple testing scripts.
• Authenticated Requests: Limited to 5,000 requests per hour per user (or GitHub App installation). For GitHub Enterprise Cloud, this can be higher (up to 15,000).

When you exceed these limits, GitHub will respond with a rate limit error. Historically, GitHub returned a 403 Forbidden for rate limit violations. However, they are increasingly standardizing on the HTTP 429 Too Many Requests status code, particularly for secondary rate limits (abuse detection mechanisms triggered by rapid, concurrent requests or CPU-intensive operations).

Typical Error Response:

{
  "message": "API rate limit exceeded for 192.0.2.1. (But here's the good news: Authenticated requests get a higher rate limit. Check out the documentation for more details.)",
  "documentation_url": "https://docs.github.com/rest/overview/resources-in-the-rest-api#rate-limiting"
}

Authentication Failures (401)

A 401 Unauthorized error means the GitHub API does not recognize your credentials. This happens when:

• You are not sending an Authorization header.
• Your Personal Access Token (PAT) has expired or been revoked.
• Your token lacks the required scopes or permissions to access the specific endpoint (e.g., trying to read a private repository with a token that only has public_repo scope).

Timeouts and Server Errors (502)

While less common, you might encounter 502 Bad Gateway, 504 Gateway Timeout, or general connection timeouts. These usually indicate:

1. GitHub Service Degredation: Check githubstatus.com to see if GitHub is experiencing an outage.
2. Expensive Queries: You are requesting a massive payload (e.g., a repository with millions of commits without proper pagination) that the API cannot process within the timeout window.

Step 1: Diagnose the Issue

Before writing fix code, you must determine exactly why your requests are failing.

Checking the Response Headers

GitHub provides crucial rate limit information in the HTTP response headers of every API request:

• x-ratelimit-limit: The maximum number of requests you're permitted to make per hour.
• x-ratelimit-remaining: The number of requests remaining in the current rate limit window.
• x-ratelimit-reset: The time at which the current rate limit window resets, in UTC epoch seconds.
• x-ratelimit-used: The number of requests you have made in the current window.

If x-ratelimit-remaining is 0, you are rate-limited. If you receive a 403 but have remaining requests, you might be hitting a secondary rate limit (abuse limit) due to concurrency.

The Rate Limit Endpoint

You can query your current limit directly without consuming a core rate limit quota:

curl -H "Accept: application/vnd.github.v3+json" \
     -H "Authorization: token YOUR_TOKEN" \
     https://api.github.com/rate_limit

This returns a JSON object detailing your core, search, and graphql limits.

Step 2: Implement Solutions

Solution 1: Authenticate Your Requests (Fixes 60/hr limit and 401s)

If you are currently making unauthenticated requests, the fastest fix is to generate a Personal Access Token (PAT) and include it in your HTTP headers.

1. Go to GitHub -> Settings -> Developer settings -> Personal access tokens.
2. Generate a new token with the minimum required scopes.
3. Add the token to your API calls via the Authorization header.

Solution 2: Implement Conditional Requests

To conserve your 5,000/hr limit, you should use conditional requests. The GitHub API supports ETag and Last-Modified headers. When you make a request, GitHub sends these headers back. On your next request to the same endpoint, include them using If-None-Match and If-Modified-Since.

If the data hasn't changed since your last request, GitHub returns an empty body with a 304 Not Modified status. Crucially, 304 responses do not count against your core rate limit.

Solution 3: Handle Secondary Rate Limits with Exponential Backoff

Secondary rate limits are triggered if you make too many concurrent requests or request CPU-intensive data too rapidly. To handle 403/429 errors gracefully, implement an exponential backoff retry mechanism.

When you receive a rate limit error, check for the Retry-After HTTP header. If present, wait exactly that many seconds before retrying. If it is not present, wait based on the x-ratelimit-reset header for primary limits, or use standard exponential backoff for 500-level errors.

Solution 4: Addressing 502 and Timeout Errors

If you consistently see 502s or timeouts:

1. Pagination: Ensure you are using pagination (?per_page=100&page=1). Requesting massive single blocks of data will overload the API and cause a timeout.
2. GraphQL Migration: If you are making multiple REST calls to assemble related data (e.g., getting a Pull Request, then iterating to get its commits and comments), switch to the GraphQL API. You can fetch exactly the nested data you need in a single, efficient request, avoiding both timeouts and rate limit exhaustion.