EngineeringIntermediate
How to Handle API Errors: HTTP Status Codes (400, 401, 429, 500) and Exponential Retries
Direct Answer & Overview
A production error-handling handbook detailing HTTP status code categories (2xx, 4xx, 5xx), rate-limit backoff algorithms (jittered exponential backoff), and circuit-breaker patterns.
1.HTTP Status Code Taxonomy: What the Numbers Mean
• 2xx Success: 200 OK (successful computation), 201 Created (resource stored), 202 Accepted (async task queued).
• 4xx Client Errors: 400 Bad Request (invalid JSON syntax), 401 Unauthorized (missing/invalid API key), 403 Forbidden (insufficient permissions), 404 Not Found (invalid endpoint path), 429 Too Many Requests (rate limit breached).
• 5xx Server Errors: 500 Internal Error (server-side crash), 502 Bad Gateway (upstream proxy failure), 503 Service Unavailable (GPU capacity temporary overload).
2.Exponential Backoff with Full Jitter
When an API returns HTTP 429 or 503, immediately retrying can trigger a 'thundering herd' retry storm that prevents the server from recovering. The industry standard is exponential backoff with randomized jitter: delay = random(0, min(max_delay, base_delay * 2^attempt)). The random jitter desynchronizes clients, allowing traffic to drain smoothly.
3.The Circuit Breaker Pattern in Mission-Critical Apps
If an external AI provider fails 5 consecutive times within 30 seconds, trip the circuit breaker: immediately fail-fast or route queries to a secondary backup provider for 60 seconds without waiting for network timeouts.
Jittered Exponential Backoff Implementation in Pythonpython
import random
import time
import requests
def call_api_with_jitter_backoff(url, payload, api_key, max_attempts=5):
headers = {"Authorization": f"Bearer {api_key}"}
for attempt in range(max_attempts):
try:
response = requests.post(url, json=payload, headers=headers, timeout=10)
if response.status_code == 200:
return response.json()
elif response.status_code in (429, 502, 503):
# Calculate exponential delay with randomized jitter
delay = random.uniform(0.5, min(10.0, 1.0 * (2 ** attempt)))
print(f"Status {response.status_code}. Retrying in {delay:.2f}s...")
time.sleep(delay)
else:
# Client error (400, 401) cannot be resolved with retry
response.raise_for_status()
except requests.exceptions.RequestException as e:
if attempt == max_attempts - 1:
raise e
time.sleep(1.0)
raise TimeoutError("Maximum retry attempts exceeded.")Frequently Asked Questions
Should I retry a 400 Bad Request error?
No. A 400 error indicates malformed payload syntax or invalid model names; retrying the identical request will always fail.
What is the Retry-After header?
When returning HTTP 429, servers often send a 'Retry-After: 5' header indicating the exact number of seconds to pause before retrying.
How does API100 handle upstream provider outages?
API100 maintains automatic multi-provider circuit breakers, routing queries to healthy backup clusters automatically.
A100
API100 Engineering Team
Infrastructure & Latency Research