Hardware /

RTX 3080 Ti

14measured results
332best gen tok/s*
12 GBreported memory

*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →

llama.cpp

ModelQuantGen tok/sPrompt tok/sSourceContext testedBuild
Llama 3.2 1B InstructQ4_K - Medium33214,470LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium32114,233LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium21212,769LocalScore ↗512+128llamafile 0.9.3Details →
Meta Llama 3.1 8B InstructQ4_K - Medium1043,739LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium84.03,407LocalScore ↗4,352llamafile 0.9.3Details →
Meta Llama 3.1 8B InstructQ4_K - Medium81.33,482LocalScore ↗512+128llamafile 0.9.3Details →
Meta Llama 3.1 8B InstructQ4_K - Medium79.23,204LocalScore ↗4,352llamafile 0.9.3Details →
Qwen2.5 14B InstructQ4_K - Medium60.22,014LocalScore ↗4,352llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium59.92,090LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium58.91,881LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium50.01,959LocalScore ↗512+128llamafile 0.9.3Details →
Qwen2.5 14B InstructQ4_K - Medium44.91,762LocalScore ↗4,352llamafile 0.9.3Details →
Qwen2.5 14B InstructQ4_K - Medium36.81,425LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium12.21,420LocalScore ↗512+128llamafile 0.9.3Details →

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.