Hardware /
GTX 1060 6GB
16measured results
86.5best gen tok/s*
6 GBreported memory
*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →
llama.cpp
| Model | Quant | Gen tok/s | Prompt tok/s | Source | Context tested | Build | |
|---|---|---|---|---|---|---|---|
| TinyLlama | Q4_0 | 86.5 | 1,782 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 78.0 | 1,875 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 77.9 | 1,885 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 74.9 | 1,828 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q8_0 | 66.8 | 1,921 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DeepSeek R1 Distill Qwen 1.5B | Q4_K - Medium | 62.5 | 1,475 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| gemma-2b-it | Q4_0 | 59.1 | 1,304 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen3 1.7B | Q4_K - Medium | 51.0 | 1,000 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Phi2 | Q4_0 | 37.6 | 760 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 3B Instruct | Q4_K - Medium | 34.0 | 743 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 3B Instruct | Q4_K - Medium | 34.0 | 703 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 3B Instruct | Q6_K | 27.9 | 682 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Phi3 | Q4_K - Medium | 27.3 | 516 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen3 4B Thinking 2507 | Q4_K - Medium | 24.6 | 542 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DeepSeek R1 Distill Qwen 7B | Q4_K - Medium | 19.1 | 365 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Mistral-7B-Instruct-v0.3 | Q4_K - Medium | 17.9 | 336 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
Ollama
No measured Ollama results yet. Submit yours →
MLX
No measured MLX results yet. Submit yours →
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.