RTX 3070 × Llama 3.2 1B Instruct

Llama 3.2 1B Instruct on RTX 3070

263best gen tok/s
8,943best prompt tok/s
640max context tested
14results

Measured · all numbers below come from the linked sources

llama.cpp

QuantGen tok/sPrompt tok/sSourceContext testedTTFTBuild
Q4_K - Medium2638,943LocalScore ↗512+128161 msllamafile 0.9.2
Q4_K - Medium2608,801LocalScore ↗512+128160 msllamafile 0.9.2
Q4_K - Medium2588,807LocalScore ↗512+128160 msllamafile 0.9.2
Q4_K - Medium2588,757LocalScore ↗512+128161 msllamafile 0.9.2
Q4_K - Medium2578,758LocalScore ↗512+128161 msllamafile 0.9.2
Q4_K - Medium2548,609LocalScore ↗512+128166 msllamafile 0.9.2
Q4_K - Medium2357,901LocalScore ↗512+128172 msllamafile 0.9.2
Q4_K - Medium2188,237LocalScore ↗512+128167 msllamafile 0.9.2
Q4_K - Medium2168,295LocalScore ↗512+128223 msllamafile 0.9.3
Q4_K - Medium2048,600LocalScore ↗512+128162 msllamafile 0.9.3
Q4_K - Medium1808,826LocalScore ↗512+128156 msllamafile 0.9.2
Q4_K - Medium1526,752LocalScore ↗512+128193 msllamafile 0.9.2
Q4_K - Medium1345,936LocalScore ↗512+128245 msllamafile 0.9.2
Q4_K - Medium1295,750LocalScore ↗512+128245 msllamafile 0.9.2

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

"Context tested" is the largest prompt+generation length actually benchmarked (tokens), not a memory-fit estimate.

What else can RTX 3070 run? →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.