How to Handle API Errors: HTTP Status Codes (400, 401, 429, 500) and Exponential Retries

Overview #

A production error-handling handbook detailing HTTP status code categories (2xx, 4xx, 5xx), rate-limit backoff algorithms (jittered exponential backoff), and circuit-breaker patterns.

HTTP Status Code Taxonomy: What the Numbers Mean #

• 2xx Success: 200 OK (successful computation), 201 Created (resource stored), 202 Accepted (async task queued).
• 4xx Client Errors: 400 Bad Request (invalid JSON syntax), 401 Unauthorized (missing/invalid API key), 403 Forbidden (insufficient permissions), 404 Not Found (invalid endpoint path), 429 Too Many Requests (rate limit breached).
• 5xx Server Errors: 500 Internal Error (server-side crash), 502 Bad Gateway (upstream proxy failure), 503 Service Unavailable (GPU capacity temporary overload).

Exponential Backoff with Full Jitter #

When an API returns HTTP 429 or 503, immediately retrying can trigger a 'thundering herd' retry storm that prevents the server from recovering. The industry standard is exponential backoff with randomized jitter: delay = random(0, min(max_delay, base_delay * 2^attempt)). The random jitter desynchronizes clients, allowing traffic to drain smoothly.

The Circuit Breaker Pattern in Mission-Critical Apps #

If an external AI provider fails 5 consecutive times within 30 seconds, trip the circuit breaker: immediately fail-fast or route queries to a secondary backup provider for 60 seconds without waiting for network timeouts.

Code Example: Jittered Exponential Backoff Implementation in Python #

import random
import time
import requests

def call_api_with_jitter_backoff(url, payload, api_key, max_attempts=5):
    headers = {"Authorization": f"Bearer {api_key}"}
    for attempt in range(max_attempts):
        try:
            response = requests.post(url, json=payload, headers=headers, timeout=10)
            if response.status_code == 200:
                return response.json()
            elif response.status_code in (429, 502, 503):
                # Calculate exponential delay with randomized jitter
                delay = random.uniform(0.5, min(10.0, 1.0 * (2 ** attempt)))
                print(f"Status {response.status_code}. Retrying in {delay:.2f}s...")
                time.sleep(delay)
            else:
                # Client error (400, 401) cannot be resolved with retry
                response.raise_for_status()
        except requests.exceptions.RequestException as e:
            if attempt == max_attempts - 1:
                raise e
            time.sleep(1.0)
    raise TimeoutError("Maximum retry attempts exceeded.")

Frequently Asked Questions #

Q: Should I retry a 400 Bad Request error?

No. A 400 error indicates malformed payload syntax or invalid model names; retrying the identical request will always fail.

Q: What is the Retry-After header?

When returning HTTP 429, servers often send a 'Retry-After: 5' header indicating the exact number of seconds to pause before retrying.

Q: How does API100 handle upstream provider outages?

API100 maintains automatic multi-provider circuit breakers, routing queries to healthy backup clusters automatically.