Hardware /

L4

7measured results
190best gen tok/s*
22 GBreported memory

*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →

llama.cpp

ModelQuantGen tok/sPrompt tok/sSourceContext testedBuild
Llama 3.2 1B InstructQ4_K - Medium1908,579LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1899,573LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1889,571LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1859,351LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium42.92,056LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium23.61,072LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium23.51,119LocalScore ↗512+128llamafile 0.9.2Details →

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.