Model monitor
Every 30 minutes, we read the OpenRouter public API: the availability, latency, and throughput of each vendor's latest models on every inference provider, plus the quantization precision each provider actually uses. A quantization change such as fp16 to fp8 is one of the most common technical reasons a "same-name model got dumber."
Updated 10/17 18:40 · Data source: OpenRouter public API
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| OpenAI (openai) | 100% | 100% | — | — | unknown |
| OpenAI (openai/flex) | — | 100% | — | — | unknown |
| Azure (azure) | — | 100% | — | — | unknown |
| Azure (azure/eu) | — | 100% | — | — | unknown |
| Azure (azure/us) | — | — | — | — | unknown |
| OpenAI (openai/fast) | — | 100% | — | — | unknown |
| OpenAI (openai/ultrafast) | — | 100% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| OpenAI (openai/flex) | 100% | 100% | — | — | unknown |
| Azure (azure) | 100% | 100% | — | — | unknown |
| OpenAI (openai) | 100% | 100% | — | — | unknown |
| Azure (azure/eu) | 100% | 100% | — | — | unknown |
| OpenAI (openai/fast) | 100% | 100% | — | — | unknown |
| Amazon Bedrock (amazon-bedrock/us-east-1) | — | 100% | — | — | unknown |
| Azure (azure/us) | — | 100% | — | — | unknown |
| OpenAI (openai/ultrafast) | — | 100% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| Google (google-vertex/global) | 100% | 100% | — | — | unknown |
| Claude Platform on AWS (claude-on-aws) | 100% | 100% | — | — | unknown |
| Azure (azure/global) | 100% | 100% | — | — | unknown |
| Anthropic (anthropic) | 100% | 100% | — | — | unknown |
| Amazon Bedrock (amazon-bedrock) | 100% | 100% | — | — | unknown |
| Google (google-vertex/europe) | 100% | 100% | — | — | unknown |
| Google (google-vertex/us) | 100% | 100% | — | — | unknown |
| Azure (azure/us) | 100% | 100% | — | — | unknown |
| Amazon Bedrock (amazon-bedrock/us-east-1) | 100% | 100% | — | — | unknown |
| Amazon Bedrock (amazon-bedrock/eu-west-1) | 100% | 100% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| Google (google-vertex/global) | 100% | 100% | — | — | unknown |
| Amazon Bedrock (amazon-bedrock) | 100% | 100% | — | — | unknown |
| Azure (azure/global) | 100% | 100% | — | — | unknown |
| Claude Platform on AWS (claude-on-aws) | 100% | 100% | — | — | unknown |
| Anthropic (anthropic) | 100% | 100% | — | — | unknown |
| Google (google-vertex/us) | 100% | 100% | — | — | unknown |
| Google (google-vertex/europe) | 100% | 100% | — | — | unknown |
| Azure (azure/us) | — | 100% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| Google AI Studio (google-ai-studio/flex) | 100% | 100% | — | — | unknown |
| Google (google-vertex/global) | 100% | 99.1% | — | — | unknown |
| Google AI Studio (google-ai-studio) | 100% | 98.0% | — | — | unknown |
| Google (google-vertex/us) | — | 100% | — | — | unknown |
| Google (google-vertex/global/priority) | — | 100% | — | — | unknown |
| Google AI Studio (google-ai-studio/priority) | — | 100% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| Google AI Studio (google-ai-studio/flex) | 100% | 100% | — | — | unknown |
| Google (google-vertex/global/flex) | 100% | 82.8% | — | — | unknown |
| Google AI Studio (google-ai-studio) | 100% | 100% | — | — | unknown |
| Google (google-vertex/global) | 100% | 100% | — | — | unknown |
| Google AI Studio (google-ai-studio/priority) | 100% | 100% | — | — | unknown |
| Google (google-vertex/global/priority) | 100% | 99.6% | — | — | unknown |
| Google (google-vertex/us) | — | 100% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| DeepSeek (deepseek) | 100% | 100% | — | — | unknown |
| InferenceNet (inference-net/fp8) | — | — | — | — | fp8 |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| OpenInference (open-inference/fp4) | 100% | 100% | — | — | fp4 |
| Relace (relace) | 100% | 100% | — | — | unknown |
| Decart (decart/fp4) | 100% | 100% | — | — | fp4 |
| Sail Research (sail-research/fp4) | 100% | 89.3% | — | — | fp4 |
| Morph (morph/fp8) | 100% | 99.9% | — | — | fp8 |
| Wafer (wafer/mxfp4) | 100% | 100% | — | — | mxfp4 |
| Cloudflare (cloudflare) | 100% | 99.9% | — | — | unknown |
| InferenceNet (inference-net/fp8) | 100% | 100% | — | — | fp8 |
| DeepInfra (deepinfra/fp8/flex) | 100% | 97.0% | — | — | fp8 |
| AtlasCloud (atlas-cloud/fp8) | 100% | 100% | — | — | fp8 |
| DeepInfra (deepinfra/fp8) | 100% | 100% | — | — | fp8 |
| StreamLake (streamlake/fp8) | 100% | 100% | — | — | fp8 |
| DeepSeek (deepseek) | 100% | 100% | — | — | unknown |
| CoreWeave (coreweave/fp8) | 100% | 88.1% | — | — | fp8 |
| GMICloud (gmicloud/fp8) | 100% | 99.3% | — | — | fp8 |
| DigitalOcean (digitalocean) | 100% | 100% | — | — | unknown |
| Novita (novita/fp8) | 100% | 100% | — | — | fp8 |
| DeepInfra (deepinfra/fp8/priority) | 100% | 99.9% | — | — | fp8 |
| Phala (phala) | 100% | 100% | — | — | unknown |
| Ionstream (ionstream) | 100% | 100% | — | — | unknown |
| Makora (makora/fp8) | 100% | 89.1% | — | — | fp8 |
| Crusoe (crusoe/fp8) | 100% | 100% | — | — | fp8 |
| Nebius (nebius/fp8) | 100% | 100% | — | — | fp8 |
| PrimeIntellect (primeintellect) | 100% | 100% | — | — | unknown |
| Baidu (baidu/fp8) | 100% | 98.3% | — | — | fp8 |
| BaseTen (baseten/us) | 100% | 100% | — | — | unknown |
| Alibaba (alibaba) | 100% | 97.4% | — | — | unknown |
| Together (together) | 100% | 100% | — | — | unknown |
| SiliconFlow (siliconflow/fp8) | 100% | 99.4% | — | — | fp8 |
| Modal (modal) | 100% | 100% | — | — | unknown |
| BaseTen (baseten) | 100% | 100% | — | — | unknown |
| Parasail (parasail/fp8) | 100% | 100% | — | — | fp8 |
| Fireworks (fireworks) | 100% | 91.0% | — | — | unknown |
| Venice (venice/fp8) | 100% | 100% | — | — | fp8 |
| Fireworks (fireworks/us) | 100% | 99.1% | — | — | unknown |
| BaseTen (baseten/fast) | 100% | 98.2% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| Relace (relace/fp4) | 100% | 100% | — | — | fp4 |
| Wafer (wafer) | 100% | 100% | — | — | mxfp4 |
| InferenceNet (inference-net/fp4) | 100% | 99.7% | — | — | fp4 |
| Wafer (wafer/us) | 100% | 100% | — | — | mxfp4 |
| Sail Research (sail-research/fp4) | 100% | 100% | — | — | fp4 |
| Morph (morph/fp8) | 100% | 99.9% | — | — | fp8 |
| Sail Research (sail-research/fp4/us) | 100% | 100% | — | — | fp4 |
| AkashML (akashml/fp4) | 100% | 98.9% | — | — | fp4 |
| Makora (makora/fp4) | 100% | 99.5% | — | — | fp4 |
| Decart (decart/mxfp4) | 100% | 99.7% | — | — | mxfp4 |
| Chutes (chutes/mxfp4) | 100% | 96.0% | — | — | mxfp4 |
| InferenceNet (inference-net/fast) | 100% | 100% | — | — | fp4 |
| Phala (phala) | 100% | 100% | — | — | unknown |
| DigitalOcean (digitalocean) | 100% | 100% | — | — | unknown |
| DeepInfra (deepinfra/mxfp4) | 100% | 100% | — | — | mxfp4 |
| Amazon Bedrock (amazon-bedrock/us-east-2) | 100% | 99.8% | — | — | unknown |
| Modal (modal/mxfp4) | 100% | 100% | — | — | mxfp4 |
| Fireworks (fireworks) | 100% | 100% | — | — | unknown |
| BaseTen (baseten/fp8) | 100% | 100% | — | — | fp8 |
| Moonshot AI (moonshotai/mxfp4) | 100% | 99.9% | — | — | mxfp4 |
| Alibaba (alibaba) | 100% | 100% | — | — | unknown |
| Fireworks (fireworks/us) | 100% | 98.7% | — | — | unknown |
| Fireworks (fireworks/fast) | 100% | 94.3% | — | — | unknown |
| Parasail (parasail/fp4) | 96.2% | 99.5% | — | — | fp4 |
| Parasail (parasail/fast) | 94.1% | 97.3% | — | — | fp4 |
| Together (together) | 93.1% | 96.5% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| Inceptron (inceptron/int4) | 100% | 100% | — | — | int4 |
| CoreWeave (coreweave/int4) | 100% | 99.7% | — | — | int4 |
| Nebius (nebius/fp4) | 100% | 96.1% | — | — | fp4 |
| Moonshot AI (moonshotai/highspeed) | 100% | 100% | — | — | int4 |
| StreamLake (streamlake) | — | 100% | — | — | unknown |
| Venice (venice/int4) | — | 100% | — | — | int4 |
| SiliconFlow (siliconflow/fp8) | — | 100% | — | — | fp8 |
| Novita (novita/int4) | — | 100% | — | — | int4 |
| Phala (phala) | — | 100% | — | — | unknown |
| GMICloud (gmicloud/fp8) | — | 100% | — | — | fp8 |
| Alibaba (alibaba/fp8) | — | 100% | — | — | fp8 |
| Cloudflare (cloudflare) | — | — | — | — | unknown |
| Moonshot AI (moonshotai/int4) | — | 100% | — | — | int4 |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| Alibaba (alibaba) | 100% | 99.9% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| Z.AI (z-ai/fp8) | 100% | 100% | — | — | fp8 |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| Alibaba (alibaba) | 100% | 99.7% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| Alibaba (alibaba) | 100% | 100% | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| xAI (xai/zdr) | 100% | 99.4% | — | — | unknown |
| xAI (xai) | 100% | 99.2% | — | — | unknown |
| xAI (xai/zdr/us) | — | 100% | — | — | unknown |
| xAI (xai/zdr/priority) | — | 100% | — | — | unknown |
| xAI (xai/priority) | — | — | — | — | unknown |
| Vendor | 30 min up | 1 days up | Latency | Throughput | Quantization |
|---|---|---|---|---|---|
| xAI (xai/zdr) | 100% | 100% | — | — | unknown |
| xAI (xai) | 100% | 100% | — | — | unknown |
| xAI (xai/zdr/us) | — | 100% | — | — | unknown |
| Amazon Bedrock (amazon-bedrock/us-west-2) | — | 97.6% | — | — | unknown |
| xAI (xai/zdr/priority) | — | — | — | — | unknown |
| xAI (xai/priority) | — | — | — | — | unknown |
Change Log
Third-party quality benchmarks
Projects that directly test how well models answer questions, each with its own methodology. We don't republish their data; we only link to them. Please visit the original sites.
An open-source, preregistered long-term benchmark that tests Claude Opus 5.5 daily on a fixed question set to see whether it has degraded since release, using exact-match scoring. The most rigorous methodology.
AI Stupid Level ↗Every 4 hours, we score models from multiple vendors on real code-fix tasks, with drift alerts and historical charts.
Is It Nerfed? ↗Continuously evaluates major models and publishes alerts when a model degrades.
BridgeBench Nerf Bench ↗Uses release day as the baseline and continuously compares the model's later performance against it.