Blog / Developer tools

API Billing Doesn't Match? How to Audit Token Counts and Charges

2026-10-18cdnping editors8 min read

When an API bill looks wrong, the platform has usually not miscalculated. More often, the way you estimated differs from the way the platform meters. Input, output, cached and reasoning tokens have different prices, different models split text with different tokenizers, and your client may be retrying requests without telling you. The most reliable way to audit a bill is to record the usage field from every response, recompute the cost with the same price list, and compare the result with the platform's itemized records.

#Why doesn't my own token estimate match the bill?

The most common cause is treating all tokens as one price. A single request involves at least four kinds of tokens, each priced separately:

Output costs much more than input, which magnifies any error. Based on prices cdnping pulled from OpenRouter's public model catalog on 2026-10-17 for 16 mainstream models, the output price is typically 3 to 5 times the input price, and for one model it is about 39 times. So the same "extra 10,000 tokens" can differ by an order of magnitude in cost depending on whether it lands on input or output. These prices change. To track changes, see the price-change log at /en/models and the line-by-line monitoring of official pricing pages at /en/changes. The method is described in How to find out right away when LLM API prices change.

#Why are agent tools especially token-hungry?

Tools like Claude Code and Codex send the full context on every turn: system instructions, tool definitions, files already read, and the results of every earlier tool call. Suppose a session runs 30 turns with an average context of 40,000 tokens. That is 1.2 million input tokens, even though the conversation you see may be only a few screens long. If none of that hits the cache, all of it is billed as normal input.

#Can different tokenizers give different token counts for the same text?

Yes. Each model family uses its own tokenizer, so the same Chinese passage can produce noticeably different token counts across models, and the same goes for code and JSON. Using one vendor's tokenizer to estimate another vendor's bill will introduce error by itself.

A more reliable approach:

• For OpenAI models you can count locally with tiktoken, but pick the right encoding and remember that the message format itself takes a few tokens.
• Anthropic provides a /v1/messages/count_tokens endpoint that returns the input token count for the actual request body, which is more accurate than a local estimate.
• For other models, trust the usage in the response. Don't infer counts with a generic tokenizer.

If you use a relay or aggregation gateway (a proxy service that forwards your requests to upstream model providers, common among developers in China), also confirm that it forwards to the model you think it does. A different model means a different tokenizer and a different price. How to verify which model an API is really giving you has consistency checks you can run yourself.

#Do streaming and retries cause duplicate charges?

There are three common sources of charges that seem to appear out of nowhere:

1. Automatic client retries: many SDKs retry timeouts, 429s and 5xx errors a few times by default. If the first request had already generated part or all of its output on the server and the response simply never reached you, that call may already have been billed, and the retry is billed again.
2. Streams cut off midway: when a streaming request is dropped partway, the tokens already generated are usually still billed. If the client then resends the whole turn, the input tokens are billed again too.
3. Replays by higher-level tools: after an error, agent tools often resend the entire context, so one failure costs "the full context plus the output already generated".

Failures and retries tend to cluster in periods when the service is unstable. Take the public request data of AGI Plan that cdnping mirrors (AGI Plan is a commercial partner of cdnping; the window is 2026-10-15 19:05 to 2026-10-17 18:05 Beijing time, with hourly counts). In the two hours at 14:05 and 15:05 on 2026-10-16 there were 722 and 877 failures, more than the successes in the same hours (664 and 455), while most other hours had failures in the double digits or fewer. If consumption is clearly high in a few specific hours during an audit, first check whether the service had an outage then, and then check your client logs to see whether the same task was sent several times. To see how availability is computed from counts like these, see Measuring AI API uptime yourself.

#How do I audit line by line with the usage field?

#Step 1: Make sure every request returns usage

Non-streaming responses include usage directly. For streaming requests, note the following:

• OpenAI-compatible APIs require "stream_options": {"include_usage": true} in the request. Only then does the last chunk carry usage; otherwise a streaming request may return no usage information at all.
• On the Anthropic API, the message_start event carries the input usage and the message_delta event carries the cumulative output tokens. Use the last message_delta; don't add up the numbers from every event.

#Step 2: Know the fields

Note that the two formats define "does input include the cache" in opposite ways. Adding cached_tokens on top of OpenAI's prompt_tokens, or treating Anthropic's input_tokens as the entire input, will both give wrong results.

#Step 3: Keep your own ledger

Add a logging layer around your calls and record one line per request. Here is a minimal example that recomputes cost from OpenAI-compatible usage:

import json, time

# Fill in prices for the channel you actually use, per million tokens
PRICE = {"input": 0.0, "cached": 0.0, "output": 0.0}

def log_usage(resp_json, request_id, path="usage.jsonl"):
    u = resp_json.get("usage") or {}
    cached = (u.get("prompt_tokens_details") or {}).get("cached_tokens", 0)
    prompt = u.get("prompt_tokens", 0)
    output = u.get("completion_tokens", 0)
    cost = ((prompt - cached) * PRICE["input"]
            + cached * PRICE["cached"]
            + output * PRICE["output"]) / 1_000_000
    row = {"ts": time.time(), "id": request_id, "model": resp_json.get("model"),
           "prompt": prompt, "cached": cached, "output": output, "cost": cost}
    with open(path, "a") as f:
        f.write(json.dumps(row) + "\n")

Use the id field from the response as request_id, or an ID you generate yourself and put in a request header. On retries, reuse the same business ID and record the attempt number separately, so you can tell from the log which requests were duplicates.

#Step 4: Reconcile with the platform's records

Once you have the platform's exported records, compare them in this order:

1. Total request count: if the platform has more entries than your log, there are calls you didn't record: SDK auto-retries, another process using the same key, or a leaked key.
2. Tokens aggregated by hour: if the difference is concentrated in a few hours, go back and check those hours for outages or dropped streams.
3. Per-request comparison: pick the requests with the largest differences, match them by request ID, and compare input, cache and output. If input matches but output differs a lot, it is most likely reasoning tokens; if input differs a lot, it is most likely the cache accounting or context growth.
4. Unit price: if the quantities all match but the amount doesn't, check whether the channel's price was changed during the period.

#What else should I watch for with relays and aggregation gateways?

With a relay or gateway, the bill you see has gone through an extra layer of conversion, so confirm the following:

• Billing unit: whether it charges per token, per request, or in a custom unit.
• Whether the conversion rules are public: whether input, output and cache are converted separately, and what the coefficients are for each model.
• How failed requests are billed: whether 5xx errors, timeouts and client-side disconnects are each charged.
• Whether you can export itemized records: with only a total and no per-request records, auditing on your own is basically impossible.

These points are covered in the 8 checks in How to choose an AI API relay.

#Example: how does a unified unit like CU convert?

AGI Plan, mentioned above (a commercial partner of cdnping), uses a unified metering unit called CU: 1 CU equals 1,000 output tokens of the baseline model, and with pay-as-you-go, 1 CNY = 500 CU. By that definition, generating 100,000 output tokens on the baseline model consumes 100 CU, equivalent to 0.2 CNY at the pay-as-you-go rate. The conversion coefficients for input tokens, cache and the other capability groups are as published at agiplan.dev/pricing; this article does not derive them on its behalf.

The advantage of a unified unit is that one quota can be switched across several capability groups. The cost is an extra conversion step before you can compare against official token prices. The audit method is the same: record usage for every request, convert it to CU using the published coefficients, and compare with the deductions in the dashboard. If you changed the endpoint address in Claude Code or Codex, confirm that requests really go where you think they do; the setup is in Custom API endpoints for Claude Code, Codex and OpenCode. Public availability for each AGI Plan capability group is on /en/status/agiplan, which you can check against when failures cluster in a period you're reconciling.

#An audit checklist

When the bill doesn't match, go through these in order:

• Whether usage reporting is enabled for streaming requests (include_usage, or reading message_delta)
• Whether input tokens mix up the "includes cache" and "excludes cache" definitions
• Whether reasoning tokens were counted as output
• How many times the SDK auto-retries, and whether your logs can distinguish retries
• Whether the difference is concentrated in a few hours, and whether the service had an outage then
• Whether the platform recorded more requests than your own log, and whether the key is used only where expected
• Whether unit prices or conversion coefficients were changed during the period
• Whether the relay or gateway offers itemized records, and whether failed requests are billed

If the audit shows that the main cost comes from context growth rather than a billing error, the fix is in how you use the API: route simple tasks such as completion and formatting to cheaper models. For the approach, see How to do multi-model routing.