Blog / Model watch

LLM Token Billing Explained: Itemizing One API Call's Bill

2026-10-22cdnping editors8 min read

The formula for LLM API token billing is simple: cost = input tokens × input price + output tokens × output price. On platforms with caching, the input tokens that hit the cache are billed separately at the cache price. The hard part is that the input token count is usually much larger than you expect, because it includes more than what you typed this turn: the system prompt, the entire conversation history and the tool definitions all count. This article starts from the usage field in the API response, itemizes the bill for one call, and ends with an estimation table you can fill in.

#What is a single call actually billed on?

Almost every major API returns a usage object in the response, and it is the raw basis for billing. Field names differ by vendor. Two layouts are common:

Two things to note. First, in the OpenAI-compatible format prompt_tokens usually already includes the cache hits, so subtract cached_tokens before multiplying by the unit prices; the Anthropic format reports the three parts separately. Second, the tokens a reasoning model "thinks" with may not be shown to you, but they are billed at the output price. The exact meaning of each field is defined by each vendor's official API docs, so check them before you integrate.

#What goes into the input tokens?

On a real bill, what you typed this turn is often a small fraction of the input. The input of a request is usually assembled from these parts:

1. System prompt: sent in full with every request. The system prompt of an agent-style tool can run to several thousand tokens.
2. Tool definitions: with function calling or MCP enabled, each tool's name, description and parameter JSON Schema counts as input. With many tools, this part can easily exceed the system prompt.
3. Conversation history: the API is stateless. The model "remembers" earlier context because the client resends every previous turn. Tool call results (such as a whole file that was read in) are part of the history too.
4. Current input: what you typed this turn, any code you pasted, and uploaded images (images are converted to tokens as well).

The output side consists of:

5. Visible output: the answer you see.
6. Reasoning output: the internal thinking of a reasoning model, billed at the output price.

#A worked example (labeled as an example)

The numbers below are hypothetical. The unit prices are $2 per million input tokens and $10 per million output tokens, a combination that falls within the range of the price snapshot cdnping took from OpenRouter's public API on October 17, 2026. The cache read price is assumed to be one tenth of the input price; check each vendor's pricing page for the real ratio.

Now suppose the first 15,000 input tokens (the system prompt, tool definitions and most of the history) hit the cache:

• Cache read: 15,000 × 0.2 / 1M = 0.003
• Uncached input: 2,000 × 2 / 1M = 0.004
• Output unchanged: 0.020
• Total 0.027, about half of the uncached cost.

In this example, the part you actually "said" this turn is only 500 tokens, less than 3% of the input.

#Why do long conversations get more expensive?

Because every turn resends the entire previous history and bills it again. Suppose the fixed prefix (system prompt + tool definitions) is 4,500 tokens and each Q&A round adds 1,000 tokens of history:

• Turn 1 input: about 4,500
• Turn 10 input: about 13,500
• Turn 20 input: about 23,500

The cumulative input over 20 turns is 4,500 × 20 + 1,000 × (0 + 1 + … + 19) = 90,000 + 190,000 = 280,000 tokens. The history portion grows quadratically: double the number of turns and the cumulative input from history nearly quadruples. A coding agent that reads files, runs commands and reads logs puts every tool result into the history, so it grows even faster.

What you can do:

• Start a new session when a task is done instead of chatting in one window all day.
• Use the client's compaction or summary feature to replace old history with a short summary.
• Enable only the tools the current task needs, to cut the tool definitions sent every time.
• Put unchanging content at the front of the request so the cache can hit. For specifics, see How to use Claude API prompt caching to save money.

#What does max_tokens control, and what doesn't it?

max_tokens (called max_output_tokens or max_completion_tokens in some APIs) caps only the output, not the input.

• What it can do: stop the model from producing an overly long output, which sets an upper bound on the output cost of a single request.
• What it can't do: reduce input cost. If the history is 100,000 tokens, the input is billed at 100,000 no matter how small max_tokens is.
• For reasoning models, reasoning tokens generally count against this limit too. If it is set too low, the model may be cut off before it finishes reasoning, leaving the visible answer empty, but the tokens already generated are still billed.
• Truncated output is billed: when finish_reason is length (or stop_reason is max_tokens), the response was truncated, and the output is charged as usual.

To control total cost, work on the input side: history length, number of tools and cache hit rate.

#How do you reconcile the bill with the usage field?

The billing dashboard on a platform is aggregated by day or month, so when something is off it is hard to pin it on a particular request. The most reliable approach is to record each call's usage in your own code.

#Step 1: Look at the raw usage of one request

curl -s https://your-api-endpoint/v1/chat/completions \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"your-model","messages":[{"role":"user","content":"Hello"}],"max_tokens":50}' \
  | jq '.usage'

Streaming requests (stream: true) may not return usage by default. OpenAI-compatible APIs usually need "stream_options": {"include_usage": true}, and usage then appears in the last data chunk.

#Step 2: Log one line per call

import json, time

PRICE = {"in": 2.0, "cache": 0.2, "out": 10.0}  # example prices, USD per million tokens; replace with your actual ones

def log_usage(model, usage, path="usage.jsonl"):
    cached = (usage.get("prompt_tokens_details") or {}).get("cached_tokens", 0)
    prompt = usage.get("prompt_tokens", 0)
    out = usage.get("completion_tokens", 0)
    cost = ((prompt - cached) * PRICE["in"] + cached * PRICE["cache"] + out * PRICE["out"]) / 1e6
    row = {"ts": int(time.time()), "model": model, "prompt": prompt,
           "cached": cached, "out": out, "est_cost": round(cost, 6)}
    with open(path, "a") as f:
        f.write(json.dumps(row) + "\n")

The Anthropic format uses input_tokens, cache_read_input_tokens, cache_creation_input_tokens and output_tokens instead, and cache writes must be priced separately at the write price.

#Step 3: Aggregate by day and compare with the dashboard

jq -s 'group_by(.ts / 86400 | floor) | map({day: (.[0].ts/86400|floor), cost: (map(.est_cost)|add), prompt: (map(.prompt)|add), out: (map(.out)|add)})' usage.jsonl

Checklist for reconciliation:

• Does your own request count match the dashboard's (retries and client resends after timeouts count as requests)?
• Did every streaming request return usage, or did some go unrecorded?
• Were cache-hit tokens billed at the cache price, and were cache writes counted separately?
• Were reasoning tokens counted as output?
• Are the unit prices current, or has the platform changed its prices?
• If the gap is large, pick a few requests and check them one by one.

If the numbers still don't match, Why your AI API bill doesn't match: token counting rules and how to check covers the common causes and a request-by-request method in more detail. Prices change; you can follow line-by-line changes to vendor pricing pages on cdnping's Changes page, and the Models page records price changes in the OpenRouter catalog. For a full tracking approach, see How to find out right away when an LLM API price changes.

#How much do unit prices differ, and does it matter?

Take the snapshot cdnping pulled from OpenRouter's public API on October 17, 2026 (a single-day snapshot, not a long-term price). Across the 16 key models we track, input prices range from $0.1 to $4 per million tokens, output prices from $0.47 to $12 per million tokens, and context windows from roughly 260K to 1.05M tokens. For the same request, choosing different models can change the cost by tens of times.

Two more patterns:

• Output is usually pricier than input. In the snapshot, most models charge 3 to 5 times as much for output as for input, and a few have a bigger gap. That is why costs climb quickly when a reasoning model "thinks a lot".
• A cheap model isn't always cheaper overall. If it takes several attempts to get it right, you pay again for the input and output of every retry.

Routing requests to models at different price points by task difficulty saves more than trimming prompts. See How to do multi-model routing for how, and How to compare LLM API prices for the pitfalls of price comparison.

#How do you convert billing units that aren't tokens?

Some platforms don't quote prices per token and use their own metering unit instead. Take AGI Plan as an example (AGI Plan is a commercial partner of cdnping): its unit is the CU, where 1 CU = 1,000 output tokens of the baseline model, and with pay-as-you-go, 1 CNY = 500 CU. How other models and input tokens convert to CU is defined at https://agiplan.dev/pricing and https://agiplan.dev/models.

The approach is the same for any such scheme: first work out whether the unit refers to input or output and which baseline model it is, then check the actual deductions against the usage in your own logs.

#Cost estimation table template

Fill in this table before launching a feature to get a rough order of magnitude.

After launch, backfill this table with the real usage from your logs during the first week. If history and tool definitions turn out larger than estimated, optimize those two first.