RTX 4080 SUPER

13measured results

247best gen tok/s*

32 GBreported memory

*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →

llama.cpp

Model	Quant	Gen tok/s	Prompt tok/s	Source	Context tested	Build
Llama 3.2 1B Instruct	Q4_K - Medium	247	17,479	LocalScore ↗	512+128	llamafile 0.9.3	Details →
Llama 2 7B	Q4_0	150	8,007	llama.cpp (Vulkan) ↗	512+128	llama.cpp Vulkan @81086cd — coopmat2	Details →
Llama 2 7B	Q4_0	148	8,125	llama.cpp (CUDA) ↗	512+128	llama.cpp CUDA @81086cd	Details →
Llama 2 7B	Q4_0	147	9,439	llama.cpp (CUDA) ↗	512+128	llama.cpp CUDA @81086cd	Details →
Llama 2 7B	Q4_0	147	7,101	llama.cpp (Vulkan) ↗	512+128	llama.cpp Vulkan @81086cd — coopmat2	Details →
Llama 3.2 1B Instruct	Q4_K - Medium	141	16,736	LocalScore ↗	512+128	llamafile 0.9.2	Details →
Meta Llama 3.1 8B Instruct	Q4_K - Medium	80.1	5,033	LocalScore ↗	512+128	llamafile 0.9.3	Details →
Meta Llama 3.1 8B Instruct	Q4_K - Medium	77.3	4,924	LocalScore ↗	512+128	llamafile 0.9.3	Details →
Qwen2.5 14B Instruct	Q4_K - Medium	57.4	2,938	LocalScore ↗	512+128	llamafile 0.9.2	Details →
Meta Llama 3.1 8B Instruct	Q4_K - Medium	54.4	4,461	LocalScore ↗	512+128	llamafile 0.9.2	Details →
Qwen2.5 14B Instruct	Q4_K - Medium	46.1	2,803	LocalScore ↗	512+128	llamafile 0.9.3	Details →
Gemma-3-12B-It	Q4_K - Medium	39.1	3,159	LocalScore ↗	512+128	llamafile 0.9.3	Details →
Qwen2.5 14B Instruct	Q4_K - Medium	29.9	2,424	LocalScore ↗	512+128	llamafile 0.9.2	Details →

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.