Blog / Developer tools

Multi-Model Routing: Cut LLM Costs by Task Without Losing Quality

2026-10-17cdnping editors7 min read

Multi-model routing means sending each request to a model tier that matches the task's difficulty. Short tasks such as completion and classification go to a lightweight tier, everyday coding goes to a mid tier, and only long-running agents and complex refactors use the flagship tier. Based on OpenRouter list prices that cdnping scraped on 2026-10-17, the unit price of the lightweight tier and the flagship tier can differ by more than 10x. If simple tasks make up most of your tokens, moving them can cut the bill noticeably. Putting a task in the wrong tier costs you quality. Below: how to split traffic, how to estimate cost, and how to check quality.

#How much do model prices differ across tiers

The table below comes from a single scrape of the OpenRouter public model catalog by cdnping on 2026-10-17 (the data source behind the model monitor at /models). Prices are in USD per million tokens. These are OpenRouter's own list prices and may not equal what the vendors charge for direct access, so check each vendor's official pricing page. Models are grouped by price range rather than listed by name; to see which models are in each range today, open /en/models.

Two things stand out:

1. The input price gap between the lightweight and flagship tiers is about 13–40x, and the output price gap is about 10–25x. If a task type passes your evals at about the same rate on a lower tier as on the flagship, that gap is money you can save.
2. Input and output must be priced separately. The outlier shows why a single blended price is misleading: if your task produces long outputs, it can cost more than the flagship tier; if you mostly read long documents and emit short answers, it is very cheap.

The snapshot is a single point in time. cdnping started collecting on 2026-10-17, so history is short. For price changes, rely on the latest records at /en/models and on vendor site changes at /changes.

#Which tier fits which task

Routing starts with classifying your own requests. The table below is a starting point; correct it with your own eval set.

A few rules of thumb:

• Split by actual request counts and token distribution, not by "importance". In coding workloads, completion and classification are frequent but use few tokens per call, while agent tasks are rare but input-heavy. Count both ends.
• You can route inside an agent too. Planning and final edits go to the flagship tier, while summarizing files after reading them, filtering logs, and parsing command output can go to the lightweight tier.
• For long Chinese text, look at the input price first. These tasks have long input and short output, so cost is driven mainly by the input price.

#How to estimate monthly cost from input and output tokens

The formula is simple:

monthly cost = Σ (monthly input tokens of that task type ÷ 1,000,000 × input price
                + monthly output tokens of that task type ÷ 1,000,000 × output price)

Here is a worked example with a hypothetical monthly workload (the numbers are illustrative and do not describe any real team):

Plugging in the prices from the snapshot:

• Everything on the flagship tier (input 2, output 10): 99M × 2 + 7.5M × 10 = 198 + 75 = about $273.
• With routing: completion and classification on the lightweight tier (0.15 / 0.6) cost about $5.7; everyday coding on the upper end of the mid tier (0.825 / 4.125) costs about $32.2; complex agents stay on the flagship tier (2 / 10) at $75; long Chinese text on the lower end of the mid tier (0.37 / 1.25) costs about $6.8. The total is about $119.7, roughly 56% less than all-flagship.

In this example the savings come mainly from two input-heavy categories: completion/classification and long Chinese text. The agent category is untouched and remains the largest single expense. So the next optimization is usually to reduce agent input, for example by trimming context, using provider-side cache pricing if available, and pushing sub-steps to a lower tier.

A script you can edit directly:

# Prices: USD per million tokens. Replace with the latest prices you find.
TIERS = {
    "light":    (0.15, 0.60),
    "mid_low":  (0.37, 1.25),    # lower end of mid tier
    "mid":      (0.825, 4.125),  # upper end of mid tier
    "flagship": (2.0, 10.0),
}

# Per task type: (requests per month, input tokens per request, output tokens per request, assigned tier)
WORKLOAD = {
    "completion": (20000, 1500, 100, "light"),
    "coding":     (3000, 8000, 1000, "mid"),
    "agent":      (100, 300000, 15000, "flagship"),
    "long_zh":    (500, 30000, 2000, "mid_low"),
}

def cost(workload, override=None):
    total = 0.0
    for name, (n, tin, tout, tier) in workload.items():
        tier = override or tier
        pin, pout = TIERS[tier]
        c = n * (tin * pin + tout * pout) / 1_000_000
        print(f"{name:<12}{tier:<10}{c:>10.2f}")
        total += c
    print(f"{'total':<22}{total:>10.2f}\n")
    return total

routed = cost(WORKLOAD)
baseline = cost(WORKLOAD, override="flagship")
print(f"Savings: {1 - routed / baseline:.0%}")

Don't guess request counts and token numbers. Pull one week from your request logs, group by task type, and take the median and P90.

#How to confirm quality has not noticeably dropped

The savings are easy to calculate; the quality loss has to be verified. Follow these steps:

1. Build a small eval set. For each task type, sample 30–100 real requests and attach a reference answer or pass criterion you trust (do the unit tests pass, is the classification correct, is the format valid).
2. Compare offline. Run the same samples through your current model and the candidate lower-tier model and record pass rates. Set an acceptable gap, for example a drop of no more than a few percentage points. That threshold depends on your business.
3. Shadow traffic. Before launch, send a small share of real requests to the lower-tier model as well, record the results without returning them, and compare.
4. Automatic escalation. When the lower-tier output fails a check (JSON parse failure, failing unit test, low confidence), retry automatically on the next tier up. Track the escalation rate; if it is too high, that task type does not belong on the lower tier.
5. Keep monitoring. Third-party inference providers may change quantization (such as fp8 or fp4), so the same model ID can behave differently over time. /en/models records each provider's quantization, availability, delistings and price changes, which can serve as an external signal.

#DIY routing or an aggregation gateway

A rough rule: if your request volume is high, task types are stable, and someone maintains the infrastructure, DIY routing gives you more control. If you are a small team that mostly uses models inside coding tools such as Claude Code or Codex and don't want to maintain several sets of keys and bills, an aggregation gateway has the lower integration cost. Either way, follow each model service's terms and local regulations; some services have regional restrictions, so go by the official documentation.

As one implementation example, AGI Plan (that partners commercially with cdnping) splits models into four capability groups: frontier reasoning (long-running agents, complex refactors), balanced general-purpose (everyday coding and code review), Chinese flagship (long Chinese text, batch jobs), and fast lightweight (completion, classification, formatting). A scheduling layer routes requests to the matching group by task difficulty, which maps to the four tiers above. It meters usage in CU, where 1 CU equals 1,000 output tokens of a baseline model, and it publishes hourly request success and failure counts per capability group (https://agiplan.dev/api/status/requests?month=YYYY-MM), which cdnping mirrors at /en/status/agiplan. The service states that it is not an officially authorized channel, so read the official site and its models page before choosing it.

#A minimal DIY routing config

If you go the DIY route, start with a simple rules file that hardcodes routing by task type. Once that runs reliably, consider a classifier that judges difficulty automatically:

routes:
  - match: { task: [completion, classify, format] }
    tier: light
    fallback: mid
  - match: { task: [code_edit, code_review, unit_test] }
    tier: mid
    fallback: flagship
  - match: { task: [agent, refactor] }
    tier: flagship
    fallback: flagship_backup   # same-tier model from another provider
  - match: { task: [long_zh, summarize] }
    tier: mid
    fallback: flagship
escalate_on: [json_parse_error, test_failed, empty_output]
max_retries: 1

Run through this checklist before going live:

• Every task type has an eval set and a pass-rate baseline
• Every tier has at least one backup provider, and you have checked recent availability of each endpoint on the API status page /en/status
• You have measured first-connection and round-trip latency from your network to each API with the online speed test; keep high-latency endpoints off latency-sensitive paths such as completion
• Logs record the tier, token counts and whether each request was escalated
• You follow /en/changes or /en/reports so you can update your price table when prices change or models are delisted

#Three things to do this week

1. Export the last week of data from your request logs and compute request counts and median tokens for the four categories above.
2. Put the numbers into the estimate script to see how much routing could save and which category is the biggest expense.
3. Move only completion and classification to the lightweight tier first, run shadow traffic for a week, and confirm the pass rate before touching the next category.