Blog / Model watch

Is the Model Dumber or the Provider Different? LLM Quantization (fp8, fp4)

2026-10-17cdnping editors7 min read

When the same model id works well one minute and badly the next, a common cause is that each request lands on a different inference provider, and providers quantize differently: some run fp8, some fp4 or int4, and many don't say. In the OpenRouter snapshot cdnping collected on 2026-10-17, only 50 of 137 provider endpoints declared a quantization level, and none declared bf16. Before you decide a model has gotten "dumber," confirm who served the request and at what precision.

#What quantization is and why it changes output

An LLM's weights are a huge pile of floating-point numbers. They are usually stored in bf16 (16-bit) after training, at 2 bytes per parameter. Quantization stores and computes them with fewer bits:

Halving the bits roughly halves memory and bandwidth pressure. The same GPU can run bigger batches, throughput goes up, and the price per token can come down. That is the direct reason providers quantize.

The cost is precision loss. Four bits can represent only 16 distinct values, so outliers (the few weights or activations with unusually large magnitude) are hard to preserve. The error in a single forward pass is small, but it gets amplified in these cases:

• Long context: error accumulates across layers and sequence length, so retrieval and citation after tens of thousands of tokens are more likely to go wrong.
• Multi-step reasoning and math: one wrong step drags the rest along.
• Strict-format output: tool-call JSON, function parameter names and code indentation are sensitive to low-probability tokens.
• Low-resource languages and proper nouns: these tokens already have low probability, and quantization makes them easier to crowd out.

Two distinctions matter. First, quantization is not the same as degradation: many open models are officially released in fp8, or as int4 with quantization-aware training (QAT), so low precision is simply the original. Second, even when two endpoints are both labeled fp8, differences in calibration data, scaling granularity, inference engine and KV cache precision can produce different results. A precision label is a clue, not a verdict.

#What precision OpenRouter providers declare

The data below comes from the cdnping models page, sourced from OpenRouter's public API. It was collected on 2026-10-17 (UTC) and is a single snapshot. cdnping only started collecting on that day, so there is no trend data yet; the numbers describe that moment only.

The snapshot covers 16 model ids and 137 provider endpoints (the same provider's different regions or tiers count as separate endpoints). Distribution of the quant field:

#Why you can't see precision for closed models

In the snapshot, Anthropic, OpenAI, Google and xAI account for 8 model ids and 57 endpoints, and every one has quant set to unknown. Closed models are deployed by the original lab or its authorized cloud partners, and the precision is not disclosed. For these models, the cause of "sometimes good, sometimes bad" has to be found elsewhere, as covered below.

#How much precision varies for open models

Of the 50 endpoints that declare a precision, 49 belong to three open models (labeled with codenames below so the table doesn't tie to specific versions):

In other words, calling one open model id on OpenRouter can be backed by a dozen to three dozen endpoints, with precision ranging from fp8 down to 4-bit. When you don't specify a provider, routing picks automatically by price, uptime and latency, so two consecutive requests hitting endpoints of different precision is perfectly normal.

A few more details from the snapshot:

• First-party endpoints can also be 4-bit. Model B on Moonshot AI's own endpoint is labeled mxfp4, and model C's first-party endpoint is labeled int4. A third party deploying at the same precision isn't automatically "cutting corners."
• One provider can offer tiers. For example, DeepInfra has three endpoints for model A: fp8, fp8/flex and fp8/priority. Same precision, different service levels.
• Uptime doesn't map to precision. The four lowest 1-day uptime endpoints for model A are fp8, unknown, fp8 and fp4, all between 88% and 90%. Look at uptime separately.

#Besides quantization, what else makes a model seem "dumber"

A model id behaving differently across endpoints can also come from:

1. Truncated context: some endpoints support a smaller maximum context than the model advertises, so the start of a long conversation gets dropped.
2. Chat template and tool-call parsing: inconsistent chat template or tool-call parser implementations show up as broken formatting, wrong parameter names, or no tool call when one is needed.
3. Default sampling parameters: providers use different default temperature and top_p, which shows in the output if you don't pass them explicitly.
4. Inference acceleration: speculative decoding, KV cache quantization and fast-style tiers can introduce subtle differences.
5. Changes at the original lab: closed-model version updates, system prompt adjustments and peak-hour policy changes. You can't see a precision field for these, only observe behavior.

#How to pin precision and debug "dumber" output

#Step 1: See which endpoints a model has and their precision

OpenRouter has a public endpoint-list API that needs no key:

# Replace <author>/<slug> with the model id you want to check
curl -s https://openrouter.ai/api/v1/models/<author>/<slug>/endpoints \
  | jq -r '.data.endpoints[] | [.provider_name, .tag, .quantization] | @tsv'

Each output row is one endpoint: provider name, endpoint tag, quantization. Field names follow OpenRouter's official docs. If you'd rather not run commands, open cdnping's models page, which shows the same data grouped by model, with each endpoint's uptime, latency and throughput.

#Step 2: Restrict precision or provider in the request

OpenRouter request bodies support a provider routing parameter. The following accepts only endpoints labeled bf16 or fp8 and does not fall back to other endpoints:

{
  "model": "<author>/<slug>",
  "messages": [{"role": "user", "content": "..."}],
  "provider": {
    "quantizations": ["bf16", "fp8"],
    "allow_fallbacks": false
  }
}

If you've already verified a particular provider's output, you can also use order to set a preferred sequence. Two caveats: when filtering by precision, endpoints that don't declare precision may be excluded, so the pool of eligible endpoints can shrink noticeably; and with fallbacks off, availability depends entirely on the endpoints you selected, so check their 1-day uptime on /en/models first.

#Step 3: Record who actually served each request

When an answer is clearly worse, grab the generation id for that request, then look up which provider served it:

curl -s "https://openrouter.ai/api/v1/generation?id=<generation_id>" \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" \
  | jq '.data | {provider_name, model, tokens_prompt, tokens_completion}'

Ideally, always log the provider name in your own logs so you have something to compare against later.

#Step 4: A/B test with fixed samples

Pick 5 to 10 prompts that represent your workload (at least one long-context prompt, one tool call and one strict-JSON output), set temperature to 0, pin two endpoints of different precision, and run each several times before comparing. A single run is dominated by randomness, so don't judge from one sample.

#Troubleshooting checklist

#Where to keep tracking precision and status changes

• /en/models: uptime, latency, throughput and quantization for the latest models on each inference provider, with automatic records of listings, delistings, price changes and quantization changes. If a provider switches an endpoint from fp8 to fp4, it leaves a record here. The page also links to several third-party "model getting dumber" benchmark projects; cdnping does not republish their data.
• /en/down/openrouter: combines official status pages, an API probe every 5 minutes and one-click user reports, with a "got dumber" option in the report choices.
• /en/changes: monitors official changelogs, pricing pages and news pages from many vendors, recording additions and removals line by line. For behavior changes in closed models, look here first for a matching announcement.
• /en/status/openrouter: probes OpenRouter's API entry point from the Cloudflare edge every 5 minutes, which helps separate "the model answered badly" from "the API itself is unstable."

If you don't call OpenRouter directly but use models through an aggregation gateway such as AGI Plan (AGI Plan is a commercial partner of cdnping), we still recommend comparing with fixed samples on a regular basis. AGI Plan's per-capability-group uptime is available at /en/status/agiplan; for specific models and capability groups, see agiplan.dev/models.

cdnping's quantization change log only started accumulating on 2026-10-17. Once the data window is longer, looking back at how often each provider adjusts precision will be more informative.