RTX 5070 Ti × Llama 3.2 1B Instruct
Llama 3.2 1B Instruct on RTX 5070 Ti
260best gen tok/s
16,482best prompt tok/s
4,352max context tested
20results
Measured · all numbers below come from the linked sources
llama.cpp
| Quant | Gen tok/s | Prompt tok/s | Source | Context tested | TTFT | Build |
|---|---|---|---|---|---|---|
| Q4_K - Medium | 260 | 15,534 | LocalScore ↗ | 512+128 | 87 ms | llamafile 0.9.2 |
| Q4_K - Medium | 254 | 15,705 | LocalScore ↗ | 512+128 | 87 ms | llamafile 0.9.2 |
| Q4_K - Medium | 240 | 16,482 | LocalScore ↗ | 512+128 | 92 ms | llamafile 0.9.2 |
| Q4_K - Medium | 240 | 16,447 | LocalScore ↗ | 512+128 | 1,240 ms | llamafile 0.9.2 |
| Q4_K - Medium | 238 | 16,091 | LocalScore ↗ | 512+128 | 1,486 ms | llamafile 0.9.2 |
| Q4_K - Medium | 238 | 16,125 | LocalScore ↗ | 512+128 | 94 ms | llamafile 0.9.2 |
| Q4_K - Medium | 123 | 14,483 | LocalScore ↗ | 512+128 | 90 ms | llamafile 0.9.2 |
| Q4_K - Medium | 118 | 13,485 | LocalScore ↗ | 512+128 | 96 ms | llamafile 0.9.2 |
| Q4_K - Medium | 110 | 10,769 | LocalScore ↗ | 512+128 | 119 ms | llamafile 0.9.3 |
| Q4_K - Medium | 107 | 13,136 | LocalScore ↗ | 512+128 | 100 ms | llamafile 0.9.2 |
| Q4_K - Medium | 106 | 10,130 | LocalScore ↗ | 512+128 | 137 ms | llamafile 0.9.2 |
| Q4_K - Medium | 100 | 11,177 | LocalScore ↗ | 512+128 | 117 ms | llamafile 0.9.3 |
| Q4_K - Medium | 98.3 | 9,569 | LocalScore ↗ | 512+128 | 131 ms | llamafile 0.9.3 |
| Q4_K - Medium | 91.0 | 9,623 | LocalScore ↗ | 512+128 | 132 ms | llamafile 0.9.3 |
| Q4_K - Medium | 87.8 | 7,272 | LocalScore ↗ | 512+128 | 179 ms | llamafile 0.9.3 |
| Q4_K - Medium | 86.8 | 7,372 | LocalScore ↗ | 512+128 | 176 ms | llamafile 0.9.3 |
| Q4_K - Medium | 53.4 | 9,413 | LocalScore ↗ | 4,352 | 144 ms | llamafile 0.9.3 |
| Q4_K - Medium | 47.7 | 9,439 | LocalScore ↗ | 4,352 | 142 ms | llamafile 0.9.3 |
| Q4_K - Medium | 46.8 | 9,444 | LocalScore ↗ | 4,352 | 143 ms | llamafile 0.9.3 |
| Q4_K - Medium | 44.0 | 8,711 | LocalScore ↗ | 512+128 | 154 ms | llamafile 0.9.3 |
Ollama
No measured Ollama results yet. Submit yours →
MLX
No measured MLX results yet. Submit yours →
"Context tested" is the largest prompt+generation length actually benchmarked (tokens), not a memory-fit estimate.
What else can RTX 5070 Ti run? →
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.