Hardware /
AMD Radeon RX 7800 XT
22measured results
125best gen tok/s*
16 GBreported memory
*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →
llama.cpp
| Model | Quant | Gen tok/s | Prompt tok/s | Source | Context tested | Build | |
|---|---|---|---|---|---|---|---|
| Llama 2 7B | Q4_0 | 125 | 2,197 | llama.cpp (Vulkan) ↗ | 512+128 | llama.cpp Vulkan @4fdbc1e | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 121 | 2,512 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 2 7B | Q4_0 | 118 | 2,017 | llama.cpp (Vulkan) ↗ | 512+128 | llama.cpp Vulkan @4fdbc1e | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 117 | 2,778 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q6_K | 112 | 3,009 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 108 | 3,091 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q8_0 | 105 | 3,073 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 3B Instruct | Q4_K - Small | 57.1 | 1,203 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| CodeLlama 7b Instruct Hf | Q4_K - Medium | 46.8 | 594 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 7B Instruct | Q4_K - Medium | 44.1 | 659 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 38.9 | 524 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 34.1 | 276 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Granite 3.1 8b Instruct | Q4_K - Medium | 30.6 | 487 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen3 14B | Q4_K - Medium | 24.1 | 324 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 23.0 | 302 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 23.0 | 302 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 23.0 | 316 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Gemma 3 12b It Heretic v2 | Q4_K - Small | 22.8 | 393 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 22.7 | 301 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 14B Instruct | Q4_K - Medium | 21.3 | 171 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Llama-3.2-1B-Instruct | Q4_0 | 13.9 | 2,435 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Gemma 3 12b It | Q5_K - Medium | 6.5 | 385 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
Ollama
No measured Ollama results yet. Submit yours →
MLX
No measured MLX results yet. Submit yours →
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.