Blog / Edge & CDN

Cloudflare for AI Apps: Caching, Rate Limiting and Origin Protection

2026-10-17cdnping editors8 min read

Putting an AI app behind Cloudflare gets you a few concrete things: static assets and a handful of read-only endpoints are cached at the edge, chat endpoints are rate limited before they reach your origin, and your origin IP is no longer exposed directly. Streaming needs one extra step. Chat endpoints must bypass the cache, response headers must go out early, and long silences need a heartbeat. Otherwise you will hit 524 timeouts or "fake streaming" where everything arrives at once. Below, in order, are configurations you can copy for caching, rate limiting, origin protection and SSE.

#Which AI app endpoints can be cached, and which cannot

Traffic in an AI app falls into the categories below, and each is handled differently. By default Cloudflare only caches static files recognized by extension. HTML and JSON responses are not cached, so the endpoints you want cached must be opted in explicitly with Cache Rules, and for the ones you don't want cached you need to make sure nobody enabled caching by accident.

To decide whether an endpoint is cacheable, ask two questions: does the response depend only on the URL, and is it fine for anyone to see the response for that URL? Cache only if both answers are yes. Be careful with model list endpoints. If one returns different available models depending on the API key, it belongs in the "per-user read-only" row.

#Cache rule examples

In the Cloudflare dashboard, go to Caching → Cache Rules and create two rules in this order:

1. Bypass cache for generation endpoints Expression: starts_with(http.request.uri.path, "/v1/chat") or starts_with(http.request.uri.path, "/v1/completions") or starts_with(http.request.uri.path, "/api/stream") Action: Bypass cache.
2. Short cache for public read-only endpoints Expression: http.request.method eq "GET" and http.request.uri.path in {"/v1/models" "/api/pricing"} Action: Eligible for cache; set Edge TTL to 60 seconds and ignore the origin's Cache-Control.

Adjust the paths for your own app. The first rule looks redundant (POST is never cached anyway), but spelling it out makes it harder for someone to break things later by adding a "cache everything under /api" rule.

#Rate limiting APIs: per IP at the edge, per user in a Worker

A single AI request costs far more than an ordinary API call, and abuse shows up on your bill, not in your bandwidth. Use two layers: a coarse per-IP limit at the edge, and a precise per-user limit in a Worker or at the origin.

#Layer 1: WAF rate limiting rules

Under Security → WAF → Rate limiting rules, create a rule such as:

• Match: http.request.method eq "POST" and starts_with(http.request.uri.path, "/v1/chat")
• Counting characteristic: IP
• Threshold: say 20 requests per 10 seconds (an example value; tune it to your real usage)
• Action: Block or Managed Challenge, for a set duration

Managed Challenge works for endpoints called from browsers. For endpoints used by SDKs or command-line tools, don't use a challenge: scripts can't complete it and will just get a pile of 403s. Return 429 directly. Available counting characteristics, time windows and rule counts depend on your plan, so check Cloudflare's official documentation.

#Layer 2: per-user limits in a Worker

Per-IP limits can't stop one account rotating through many IPs, and they hurt users behind a shared egress such as a company or school network. Limiting by API key or user ID is more accurate, and the Workers Rate Limiting binding does it directly:

# wrangler.toml
name = "ai-edge"
main = "src/index.js"
compatibility_date = "2026-10-01"

[[ratelimits]]
name = "USER_LIMITER"
namespace_id = "1001"
simple = { limit = 30, period = 60 }  # 30 requests per key per 60 seconds, example value

// src/index.js
export default {
  async fetch(request, env) {
    const url = new URL(request.url);

    if (url.pathname.startsWith("/v1/chat")) {
      const auth = request.headers.get("Authorization") || "";
      if (!auth) return new Response("missing key", { status: 401 });

      // Use a hash of the key as the rate limit key so the plaintext key never shows up in logs
      const digest = await crypto.subtle.digest("SHA-256", new TextEncoder().encode(auth));
      const key = btoa(String.fromCharCode(...new Uint8Array(digest))).slice(0, 24);

      const { success } = await env.USER_LIMITER.limit({ key });
      if (!success) {
        return new Response(JSON.stringify({ error: "rate_limited" }), {
          status: 429,
          headers: { "Content-Type": "application/json", "Retry-After": "60" },
        });
      }
    }

    // Forward to the origin with a shared secret that only the edge knows
    const upstream = new Request(request);
    upstream.headers.set("X-Edge-Secret", env.EDGE_SECRET);
    const resp = await fetch(upstream);

    // Hand the body stream straight back. Don't await resp.text(), or streaming turns into a single response
    return new Response(resp.body, resp);
  },
};

This binding's counts are approximate per edge location. That is good enough to block abuse but not for exact billing. Limits that must be accurate, such as quotas and balances, still belong in your origin or database. Set EDGE_SECRET with wrangler secret put EDGE_SECRET, and keep it out of your code repository.

For sign-up and free-trial entry points, you can also add Turnstile bot verification together with email verification. That stops bulk registration better than IP rate limiting alone.

#Making your origin accept only Cloudflare traffic

If you only put your domain on Cloudflare and leave the origin open, an attacker who finds your origin IP can bypass every edge rule and hit it directly. The approaches below can be combined, listed from lowest to highest effort:

IP allowlists have a loophole that is often overlooked: every Cloudflare customer's traffic comes from the same IP ranges, so pair the allowlist with a shared secret or mTLS. Also, before changing your origin IP, check historical DNS records and email headers for leaks of the old IP. Once an old IP has leaked, putting the site behind Cloudflare does not hide it.

#Why SSE streaming stalls or times out

Three problems are common with streaming chat behind the Cloudflare proxy.

#1. Buffering turns the stream into one response

Cloudflare's proxy forwards text/event-stream as it receives it, so the problem is usually elsewhere in the chain:

• Nginx in front of your origin buffers upstream responses by default. Add proxy_buffering off; in the relevant location, or have the app return an X-Accel-Buffering: no response header.
• Compression middleware in your app framework may wait until it has enough data to flush. Turn compression off on streaming routes.
• Send Content-Type: text/event-stream and Cache-Control: no-cache, no-transform in the response headers.
• If a Worker reads the whole body (await resp.text(), resp.json()) before returning, streaming breaks the same way. To rewrite content, process it chunk by chunk with a TransformStream.

#2. 524 timeouts

Cloudflare waits only so long for the origin to respond. The limit is 100 seconds on the free plan and most paid plans (check the official documentation), after which it returns a 524. For AI apps the risk is the time before the first token: a long context or a long reasoning phase can leave the connection silent for a while. The fix is to return 200 and the response headers as soon as the request arrives, send an SSE comment line first, and then send a heartbeat every 15 to 30 seconds:

: ping

A line starting with a colon is an SSE comment, and standards-compliant clients ignore it. For non-streaming jobs that genuinely run longer than the limit, switch to "submit a job and poll for the result" or use a queue. Don't count on extending the timeout.

#3. The upstream keeps running after the client disconnects

When a user closes the page, the edge connection drops, but the origin's request to the upstream model may keep generating and keep billing. Have the origin listen for the connection-close event and cancel the upstream request. In Node.js, for example, pass an AbortController signal to the upstream fetch and call abort in the request's close event.

#Pre-launch checklist

After setup, confirm each item with the commands below. It takes a few minutes:

# 1. Are static assets served from cache? (the second request should show cf-cache-status: HIT)
curl -sI https://your.app/assets/app.3f9c1a.js | grep -i cf-cache-status

# 2. Does the chat endpoint bypass the cache? (should be BYPASS or DYNAMIC, never HIT)
curl -sI -X POST https://your.app/v1/chat -H "Authorization: Bearer test" | grep -i cf-cache-status

# 3. Does the stream arrive chunk by chunk? (-N disables curl's own buffering; output should appear line by line)
curl -N https://your.app/v1/chat -H "Authorization: Bearer $KEY" \
  -H "Content-Type: application/json" -d '{"stream":true,"messages":[{"role":"user","content":"数到 20"}]}'

# 4. Is rate limiting working? (send repeated requests; later ones should return 429)
for i in $(seq 1 40); do curl -s -o /dev/null -w "%{http_code}\n" -X POST https://your.app/v1/chat -H "Authorization: Bearer test"; done | sort | uniq -c

# 5. Does the origin refuse direct connections? (should time out, return 403, or refuse the connection)
curl -sI --resolve your.app:443:<origin-ip> https://your.app/

#What Cloudflare acceleration does not cover

The edge can only optimize the "user to edge" and "edge to your origin" legs. The leg where your origin calls the model API still depends on where the origin is. For reference, cdnping probed official API endpoints from Cloudflare's Denver (DEN) node on October 17, 2026 (UTC), once every 5 minutes over a window of about 85 minutes that day (source: /en/status). The median was 73 ms for api.anthropic.com, 96 ms for api.openai.com, 353 ms for api.deepseek.com, and 395 ms for open.bigmodel.cn (Zhipu). The window is short, so it only indicates a direction: deploy your origin close to the upstream you depend on most, because the edge layer can't replace that choice. If your origin calls an aggregation gateway such as AGI Plan (AGI Plan is a commercial partner of cdnping), you can see its probes and public availability at /en/status/agiplan.

Access from mainland China is a separate practical issue. Latency over Cloudflare's international network varies a lot by carrier and region, and using in-country nodes inside mainland China requires an ICP filing (China's mandatory website registration) and access through a licensed provider. Before you commit, use cdnping's online speed test to see connect and round-trip times from your network to each CDN and API, then see the CDN speed ranking, how to choose a public CDN in China and AI API latency from China.

#Next steps

• Using the table in the first section, classify your routes one by one and write the two cache rules.
• Add a WAF rate limiting rule for chat endpoints, then add the per-key Worker limit shown above.
• Run through the checklist, paying special attention to streaming output and direct origin access.
• If you'd rather not maintain this configuration yourself, cdnping Edge offers edge acceleration on Cloudflare's global network, quoted on request, with a reply within 1 business day after you submit your email. Any acceleration inside mainland China also requires an ICP filing.