RTX 5080

13measured results

192best gen tok/s*

15 GBreported memory

*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →

llama.cpp

Model	Quant	Gen tok/s	Prompt tok/s	Source	Context tested	Build
Llama 2 7B	Q4_0	192	8,055	llama.cpp (Vulkan) ↗	512+128	llama.cpp Vulkan @f6b533d — coopmat2	Details →
Llama 2 7B	Q4_0	185	7,445	llama.cpp (Vulkan) ↗	512+128	llama.cpp Vulkan @f6b533d — coopmat2	Details →
Llama 2 7B	Q4_0	185	9,488	llama.cpp (CUDA) ↗	512+128	llama.cpp CUDA @8a4280c	Details →
Llama 2 7B	Q4_0	182	8,297	llama.cpp (CUDA) ↗	512+128	llama.cpp CUDA @8a4280c	Details →
Llama 3.2 1B Instruct	Q4_K - Medium	103	13,606	LocalScore ↗	512+128	llamafile 0.9.2	Details →
Meta Llama 3.1 8B Instruct	Q4_K - Medium	44.9	4,428	LocalScore ↗	512+128	llamafile 0.9.2	Details →
Llama 3.2 1B Instruct	Q4_K - Medium	41.7	9,611	LocalScore ↗	512+128	llamafile 0.9.3	Details →
Llama 3.2 1B Instruct	Q4_K - Medium	41.2	9,533	LocalScore ↗	512+128	llamafile 0.9.3	Details →
Llama 3.2 1B Instruct	Q4_K - Medium	41.1	9,524	LocalScore ↗	512+128	llamafile 0.9.3	Details →
Llama 3.2 1B Instruct	Q4_K - Medium	41.0	9,253	LocalScore ↗	512+128	llamafile 0.9.3	Details →
Qwen2.5 14B Instruct	Q4_K - Medium	28.9	1,725	LocalScore ↗	512+128	llamafile 0.9.2	Details →
Qwen2.5 14B Instruct	Q4_K - Medium	23.9	2,275	LocalScore ↗	512+128	llamafile 0.9.2	Details →
Qwen2.5 14B Instruct	Q4_K - Medium	23.7	2,291	LocalScore ↗	512+128	llamafile 0.9.2	Details →

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.