of Nemotron 3.5 Lightning's tool calls fail on DeepInfra (bf16).
Updated October 29, 2026
Tool-call index
Not all models are served equally by all providers. On a typical model, the worst provider breaks tool calls 19× as often as the best. Find the best one for your agents.
| # | Model | Broken tool calls | Rank | Best on the model | Tool calls a day |
|---|---|---|---|---|---|
| 1 | Kimi K3mxfp4 | 0.01% | 1 of 24 | — | 66k |
| 2 | Hy4 previewfp4 | 0.10% | 1 of 6 | — | 90k |
| 3 | Nemotron 3 Ultrafp4/flex | 0.17% | 1 of 4 | — | 1.6k |
| 4 | GLM 5.3 Flashfp4 | 0.27% | 8 of 34 | NextBit (fp4) 0.00% | 228k |
| 5 | DeepSeek V4.1 Flash 1017fp8/flex | 0.27% | 7 of 35 | Sail Research (fp4) 0.03% | 317k |
| 6 | DeepSeek V4.1 Flashfp8/priority | 0.30% | 6 of 35 | Prime Intellect 0.06% | 104k |
| 7 | GLM 5.2fp4 | 0.40% | 6 of 27 | Friendli 0.15% | 35k |
| 8 | DeepSeek V4.1 Flash 1017fp8 | 0.49% | 14 of 35 | Sail Research (fp4) 0.03% | 2.0M |
| 9 | Kimi K2.6fp4 | 0.51% | 4 of 14 | SiliconFlow (fp8) 0.06% | 23k |
| 10 | DeepSeek V4 Pro 0813fp8 | 0.60% | 7 of 21 | Ionstream 0.04% | 3.7k |
| 11 | DeepSeek V4.1 Flashfp8 | 0.64% | 18 of 35 | Prime Intellect 0.06% | 2.2M |
| 12 | DeepSeek V3.2fp4 | 0.65% | 2 of 4 | SiliconFlow (fp8) 0.60% | 67k |
| 13 | Qwen3.8 27Bbf16 | 0.66% | 8 of 21 | Darkbloom (fp4) 0.07% | 129k |
| 14 | DeepSeek V4.1 Flash 1017fp8/priority | 0.71% | 20 of 35 | Sail Research (fp4) 0.03% | 961k |
| 15 | Qwen3.5-9Bbf16 | 0.84% | 4 of 5 | Darkbloom (fp4) 0.08% | 131k |
| 16 | GLM 5.3fp4 | 0.99% | 26 of 42 | Amazon Bedrock 0.08% | 42k |
| 17 | Nemotron 3 Ultrafp4 | 1.07% | 3 of 4 | DeepInfra (fp4/flex) 0.17% | 7.7k |
| 18 | Qwen3 235B A22B Instruct 2507fp8 | 1.09% | 2 of 3 | Parasail (fp8) 0.78% | 76k |
| 19 | Qwen3.5 397B A17Bfp8 | 1.13% | 3 of 7 | Alibaba Cloud Int. 0.58% | 9.5k |
| 20 | Gemma 4 31Bfp4/turbo | 1.17% | 4 of 12 | ModelRun [by Modular] (fp4) 0.09% | 110k |
| 21 | Gemma 4 26B A4Bfp8 | 1.20% | 6 of 11 | NextBit (bf16) 0.00% | 144k |
| 22 | MiniMax M3fp8 | 1.43% | 5 of 11 | ModelRun [by Modular] (fp8) 0.27% | 4.5k |
| 23 | gpt-oss-120bbf16 | 1.46% | 2 of 15 | Groq 0.04% | 66k |
| 24 | MiMo-V2.6-Profp8 | 1.48% | 1 of 4 | — | 244k |
| 25 | DeepSeek V4 Flash 0423fp8 | 1.50% | 8 of 16 | Crusoe (fp8) 0.46% | 443k |
| 26 | DeepSeek V4 Pro 0423fp8 | 1.69% | 7 of 13 | Cloudflare 0.78% | 13k |
| 27 | DeepSeek V4 Flash Vision Expfp8 | 1.92% | 3 of 3 | GMICloud (fp8) 0.58% | 83k |
| 28 | Granite 4.2 8Bbf16 | 2.00% | 1 of 2 | — | 4.5k |
| 29 | gpt-oss-20bbf16 | 2.01% | 3 of 7 | Darkbloom (fp8) 0.13% | 5.3k |
| 30 | Qwen3.6 35B A3Bfp8 | 2.01% | 3 of 6 | AkashML (fp8) 1.63% | 24k |
| 31 | MiMo-V2.6-Flashfp8 | 2.12% | 3 of 9 | Venice (fp8) 1.41% | 178k |
| 32 | GLM 4.7fp4 | 2.22% | 3 of 3 | Z.ai (fp4) 0.93% | 2.3k |
| 33 | Gemma 4 31Bfp8 | 2.33% | 9 of 12 | ModelRun [by Modular] (fp4) 0.09% | 51k |
| 34 | DeepSeek V4.1 Flashfp8/flex | 3.20% | 34 of 35 | Prime Intellect 0.06% | 13k |
| 35 | DeepSeek V3.1fp4 | 4.03% | 3 of 3 | SiliconFlow (fp8) 1.04% | 14k |
| 36 | DeepSeek V4 Flash 0731fp8 | 4.03% | 19 of 22 | Sail Research (fp4) 0.04% | 587k |
| 37 | Nemotron 3.5 Super VLfp4 | 7.95% | 7 of 8 | Together 0.66% | 24k |
| 38 | Llama 3.3 70B Instructfp8/turbo | 9.94% | 3 of 5 | Groq 2.21% | 6.5k |
| 39 | Nemotron 3.5 Lightningbf16 | 9.96% | 4 of 4 | Darkbloom (int4) 0.01% | 21k |
Get updates when a new model drops
And which provider to use.
For inference providers
Tool calling doesn't have to break
The same provider can handle tool calls well on one model and struggle on the next.
on Kimi K3, its best model.
models, where the settings tell OpenRouter to skip DeepInfra.