Hardware /
RTX 3090
113measured results
377best gen tok/s*
24 GBreported memory
*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →
llama.cpp
| Model | Quant | Gen tok/s | Prompt tok/s | Source | Context tested | Build | |
|---|---|---|---|---|---|---|---|
| Llama 3.2 1B Instruct | Q4_K - Medium | 377 | 15,607 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 368 | 15,252 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 367 | 15,400 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 361 | 14,974 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 360 | 15,064 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 350 | 14,749 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 348 | 14,452 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 342 | 13,768 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 342 | 14,411 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 340 | 13,805 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 339 | 14,601 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 331 | 13,990 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 330 | 14,061 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 328 | 14,169 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 321 | 13,357 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 319 | 13,125 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 308 | 12,344 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 301 | 12,964 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q6_K | 300 | 13,357 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 0.5B Instruct | Q6_K | 300 | 19,958 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 276 | 12,950 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 216 | 12,560 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 208 | 12,930 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Qwen2.5 1.5B Instruct | Q6_K | 207 | 11,291 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 1.5B Instruct | Q6_K | 205 | 11,146 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 1.5B Instruct | Q6_K | 204 | 11,246 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Gemma 3 1b It | Q4_K - Medium | 199 | 17,009 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 194 | 13,974 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 193 | 14,107 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 186 | 13,196 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 176 | 12,960 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 2 7B | Q4_0 | 172 | 5,201 | llama.cpp (Vulkan) ↗ | 512+128 | llama.cpp Vulkan @d05fe1d — coopmat2 | Details → |
| Llama 2 7B | Q4_0 | 164 | 4,666 | llama.cpp (Vulkan) ↗ | 512+128 | llama.cpp Vulkan @d05fe1d — coopmat2 | Details → |
| Llama 2 7B | Q4_0 | 162 | 5,560 | llama.cpp (CUDA) ↗ | 512+128 | llama.cpp CUDA @c76b420 | Details → |
| Llama 2 7B | Q4_0 | 158 | 5,175 | llama.cpp (CUDA) ↗ | 512+128 | llama.cpp CUDA @c76b420 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 151 | 11,382 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Phi 4 Mini Instruct | Q4_K - Medium | 145 | 5,535 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Phi 3 Mini 128k Instruct | Q6_K | 132 | 4,481 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Phi 3.5 Mini Instruct | Q6_K | 131 | 4,403 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 117 | 3,895 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 117 | 3,926 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Qwen3-4B | Q4_K - Medium | 117 | 4,467 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 115 | 11,478 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 114 | 11,568 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 113 | 3,991 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 112 | 12,246 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 110 | 3,895 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 110 | 3,898 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 109 | 3,858 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 105 | 11,674 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 103 | 3,552 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 103 | 3,637 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 103 | 3,511 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 101 | 3,454 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 101 | 3,307 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 85.4 | 3,876 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| EuroLLM 9B Instruct | Q4_K - Medium | 83.9 | 2,827 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 83.1 | 3,641 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 79.7 | 3,363 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Meta Llama 3.1 8B Instruct Abliterated | Q4_K - Medium | 77.6 | 2,918 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Granite 3.2 8b Instruct | Q6_K | 73.7 | 2,744 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 68.1 | 2,187 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Qwen3 4B | Q4_K - Medium | 66.1 | 3,933 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 65.4 | 2,251 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen3 4B | Q4_K - Medium | 65.2 | 4,397 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 65.1 | 2,185 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 63.7 | 2,135 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 63.6 | 2,151 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 63.4 | 2,131 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 62.0 | 2,126 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 61.9 | 2,003 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 61.8 | 2,106 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 61.6 | 2,083 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 61.0 | 1,917 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 60.6 | 1,911 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 60.5 | 2,034 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 59.9 | 2,005 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 59.8 | 2,030 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 59.8 | 2,010 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 59.7 | 1,837 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen3 8B | Q4_K - Medium | 59.7 | 2,970 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 59.6 | 2,049 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 59.4 | 1,944 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 59.1 | 1,876 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 59.0 | 2,007 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 58.6 | 1,878 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 58.4 | 1,873 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 58.4 | 1,938 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 58.3 | 1,854 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 58.2 | 1,821 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 56.7 | 1,877 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 55.5 | 1,884 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 55.2 | 1,728 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gemma Ifable | Q4_K - Medium | 54.7 | 2,318 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 53.2 | 1,882 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 52.3 | 1,719 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 52.1 | 1,705 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 50.7 | 2,189 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AReaL Boba RL 7B | F16 | 50.2 | 3,754 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 48.7 | 1,957 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 48.7 | 1,943 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 48.1 | 1,916 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 48.0 | 1,923 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 45.3 | 8,635 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Qwen3 30B A3B | Q4_K - Medium | 43.2 | 814 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 40.9 | 1,852 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen3 14B | Q3_K - Large | 39.3 | 1,604 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| QwQ 32B | Q4_K - Medium | 30.6 | 915 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 27.1 | 1,692 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen3-30B-A3B-Instruct-2507 | Q4_K - Medium | 27.0 | 579 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 25.1 | 2,540 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Fallen Gemma3 27B v1 | Q4_K - Medium | 25.1 | 835 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 13.9 | 1,383 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
Ollama
No measured Ollama results yet. Submit yours →
MLX
No measured MLX results yet. Submit yours →
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.