Blog / Troubleshooting

429 Too Many Requests: How to Fix LLM API Rate Limit Errors

2026-10-23cdnping editors8 min read

A 429 Too Many Requests from an LLM API means you have used more than your quota in some time window, and the server is declining this request for now. It is usually not an outage and not a network problem. The order of operations: look at the response headers and error body first to confirm whether you hit the request limit (RPM), the token limit (TPM), or your account quota. If it is one of the first two, wait for Retry-After or the reset time, retry, and add client-side rate limiting. If your quota is exhausted, no number of retries will help; you need to add credit or change plans.

Most generic 429 explainers only say "too many requests," but LLM APIs add a second dimension: tokens. One request carrying 100,000 tokens of context can fill your TPM in a single call while RPM is nowhere near its cap. The sections below follow the order in which you should investigate.

#What are the different sources of a 429?

429s from LLM APIs fall into three groups, and each needs a different fix:

One more case gets mixed in: some platforms count usage across the whole organization or project. If one API key is used by several machines or several teammates, each may look quiet on its own while the total is over the limit. When investigating, add up every caller under the same organization.

#How do I tell which limit I hit from the response headers?

The most direct way is to print the full response headers of one request. The commands below use -i to show headers, with a tiny input so they barely use any quota:

# OpenAI-format endpoint; replace $MODEL with a model your account can use
curl -sS -i https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"model\":\"$MODEL\",\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}],\"max_tokens\":1}" \
  | grep -iE '^(HTTP|x-ratelimit|retry-after)'

# Anthropic-format endpoint
curl -sS -i https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "content-type: application/json" \
  -d "{\"model\":\"$MODEL\",\"max_tokens\":1,\"messages\":[{\"role\":\"user\",\"content\":\"hi\"}]}" \
  | grep -iE '^(HTTP|anthropic-ratelimit|retry-after)'

Successful requests carry these headers too, so you do not have to wait for a 429 to look. Here is what the two providers send. Check the official docs for exact field names, since providers change them:

The rule is simple: whichever remaining is near 0 is the limit you hit. If remaining-requests is high but remaining-tokens is 0, it is TPM. If Anthropic's input tokens run out first, your context is too long, not your output.

The error body is the second clue. In OpenAI-format errors, code set to rate_limit_exceeded is a rate limit, while insufficient_quota means your quota is used up. The latter is also a 429, but waiting will not fix it. Anthropic's 429 body has type set to rate_limit_error, and message says which limit was hit. Gemini's 429 status is RESOURCE_EXHAUSTED, and details may include a suggested retry delay. Aggregation gateways and relay services may pass upstream headers through unchanged, rewrite them, or drop them entirely, so if you go through a gateway, read the gateway's own docs first.

#How should I use Retry-After?

The rule: if Retry-After is present, follow it; if not, use the reset time; if neither exists, use exponential backoff. A few details that trip people up:

1. Units. Per the HTTP spec, Retry-After can be a number of seconds or an HTTP date. LLM APIs usually send seconds, but your parser should handle both.
2. Zero or tiny values. You will sometimes see retry-after: 0, and retrying immediately will probably hit another 429. Set a floor, say 1 second, and add random jitter.
3. Missing header. Some gateways, proxies, and SDK wrappers strip it. Fall back to x-ratelimit-reset-*, and if that is missing too, use exponential backoff: 1, 2, 4, 8 seconds, capped at around 60, plus 0 to 1 second of random jitter so multiple processes do not wake up together and hit the wall again.
4. Cap the number of retries. If you still get 429 after 3 to 5 attempts, stop retrying. Surface the error and let a higher layer handle it or queue the work.
5. Do not retry quota-type 429s. If the code is insufficient_quota or the message mentions billing or credit, fail immediately with a clear message. Retrying just wastes time.

Here is a Python helper that does not depend on any particular SDK:

import random, re, time
from email.utils import parsedate_to_datetime

def parse_duration(s):
    """Parse OpenAI-style durations such as '6m0s', '1.5s', '20ms'."""
    total = 0.0
    for num, unit in re.findall(r"([\d.]+)(ms|s|m|h)", s or ""):
        total += float(num) * {"ms": 0.001, "s": 1, "m": 60, "h": 3600}[unit]
    return total or None

def wait_seconds(headers, attempt):
    h = {k.lower(): v for k, v in headers.items()}
    ra = h.get("retry-after")
    if ra:
        try:
            sec = float(ra)
        except ValueError:  # HTTP date format
            sec = parsedate_to_datetime(ra).timestamp() - time.time()
        return max(sec, 1.0) + random.random()
    for key in ("x-ratelimit-reset-tokens", "x-ratelimit-reset-requests"):
        sec = parse_duration(h.get(key))
        if sec:
            return max(sec, 1.0) + random.random()
    return min(2 ** attempt, 60) + random.random()

def call_with_retry(send, max_attempts=5):
    for attempt in range(max_attempts):
        resp = send()
        if resp.status_code != 429:
            return resp
        body = resp.text.lower()
        if "insufficient_quota" in body or "billing" in body:
            raise RuntimeError("Quota or balance exhausted, retrying won't help: " + resp.text[:200])
        time.sleep(wait_seconds(resp.headers, attempt))
    raise RuntimeError("Repeated 429s, giving up")

send is a function that makes the request and returns a requests.Response. If you use an official SDK, it probably retries on its own. Check how many retries it does by default, so your own retry loop does not wrap around the SDK's and multiply the attempts.

#How do I rate-limit on the client instead of waiting for a 429?

Retrying is only a safety net. For batch jobs it is better to pace requests to your limits so 429s almost never happen. Because LLM APIs limit both requests and tokens, your limiter needs two buckets working together:

import threading, time

class DualBucket:
    """Rate-limit by requests per minute and tokens per minute at the same time."""
    def __init__(self, rpm, tpm):
        self.cap = {"req": rpm, "tok": tpm}
        self.level = dict(self.cap)
        self.rate = {"req": rpm / 60, "tok": tpm / 60}
        self.t = time.monotonic()
        self.lock = threading.Lock()

    def acquire(self, est_tokens):
        est_tokens = min(est_tokens, self.cap["tok"])  # avoid waiting forever when one request exceeds bucket capacity
        while True:
            with self.lock:
                now = time.monotonic()
                for k in self.level:
                    self.level[k] = min(self.cap[k], self.level[k] + (now - self.t) * self.rate[k])
                self.t = now
                if self.level["req"] >= 1 and self.level["tok"] >= est_tokens:
                    self.level["req"] -= 1
                    self.level["tok"] -= est_tokens
                    return
            time.sleep(0.05)

bucket = DualBucket(rpm=50, tpm=40_000)   # example numbers; use your account's actual limits
# Before each request: bucket.acquire(estimated input tokens + max_tokens)

A few practical notes:

• Set rpm and tpm to 80 to 90 percent of your real limits, leaving headroom for other processes and estimation error.
• Include max_tokens in the token estimate. Some platforms reserve output capacity against max_tokens when the request starts, so setting it far above what you need wastes TPM.
• Share one bucket across processes and machines. An in-memory bucket on one host cannot control other hosts. Put it in shared storage such as Redis, or route all calls through a single egress service.
• Calibrate with the response headers. After each response, read remaining. If it is much lower than your bucket predicts, someone else is sharing the quota.

#How do I reduce 429s at the source?

If you still hit TPM often after pacing, work on the requests themselves:

• Shorten the context. Agent-style tools tend to stuff entire files or directories into the prompt. Limit how much is read in one go, or summarize before asking.
• Use prompt caching. A long, repeated system prompt can be cached. According to Anthropic's docs, for some models, cache-read tokens do not count toward the input-token limit. For the rules and how to set it up, see How to use Claude API prompt caching to save money.
• Move light tasks to a different model. Classification, formatting, and similar jobs can go to a lightweight model, which usually has its own separate limits, leaving your main model's TPM for hard tasks. For the approach, see How to do multi-model routing.
• Request a higher limit. Most platforms raise limits as your usage tier goes up, and the console shows your current tier and the upgrade conditions. Go through the official channel. Do not stack limits by opening multiple accounts or rotating keys, which violates most terms of service.

#What is the difference between 429, 529, and 503?

This gets its own section because many people mistake server-side overload for hitting their own limit, then change code and cut concurrency when the problem is not on their side.

The headers are the key to telling them apart. On a real 429, your remaining is usually at the floor. If remaining is plentiful and you still get 429, or a large number of unrelated users report rate limiting at the same time, the server is more likely tightening limits or having trouble. For the full meaning of each error code and which ones to retry, see the LLM API error code cheat sheet.

#How can I tell whether many people are being rate-limited at once?

If your own limits are fine but 429s keep coming, check the following:

1. Open cdnping's Is it down? page and find the service, such as Claude or ChatGPT. The page combines the official status page, an API probe every 5 minutes, and one-click user reports, which include a "rate limited" report type. If rate-limit reports have clearly risen recently, you are not the only one. You can submit one too.
2. Check the API status page, for example /en/status/anthropic and /en/status/openai. It probes the official API endpoints from the Cloudflare edge every 5 minutes and syncs official status page incidents, so you can see whether the endpoint itself is unhealthy.
3. When reading an official status page, pay attention to the incident level and affected components. A minor incident such as "elevated error rates on some models" often shows up as more 429s or 529s. For how to read them, see How to read the OpenAI and Claude status pages.

If you use a subscription-metered tool like Codex, its "usage limit" and an API 429 are two separate mechanisms. When your allowance comes back depends on the platform's announcements; see the Codex usage limit reset timeline.

#A checklist you can follow

When you hit a 429, check in this order:

1. Print the response headers and error body to confirm whether it is RPM, TPM, or quota.
2. If the code is insufficient_quota or the message mentions billing, stop retrying and check your balance and spend cap in the console.
3. If retry-after is present, wait that long; if it is 0, wait at least 1 second and add jitter.
4. If there is no retry-after, read x-ratelimit-reset-* or anthropic-ratelimit-*-reset, otherwise use exponential backoff with a retry cap.
5. Find out who else is calling under the same organization and key, and count them in the total.
6. Check the SDK's built-in retries so retries do not stack.
7. For batch jobs, add dual-bucket rate limiting, and include max_tokens in the token estimate.
8. If your limits are ample and you still get 429s, look at the "rate limited" reports at /en/down and the probes and official incidents at /en/status to see whether the problem is on the server side.