Hardware /

Apple M4 Max (32-core GPU)

34measured results
171best gen tok/s*
36 GBreported memory

*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →

llama.cpp

ModelQuantGen tok/sPrompt tok/sSourceContext testedBuild
Llama 3.2 1B InstructQ4_K - Medium1713,301LocalScore ↗4,352llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1693,211LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1673,325LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1653,316LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1653,347LocalScore ↗4,352llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1653,234LocalScore ↗4,352llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1653,112LocalScore ↗4,352llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1643,224LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1623,197LocalScore ↗4,352llamafile 0.9.2Details →
DeepSeek R1 Distill Qwen 1.5BQ4_K - Medium1332,424LocalScore ↗512+128llamafile 0.9.2Details →
Llama 2 7BQ4_070.0714llama.cpp (Metal) ↗512+128llama.cpp MetalDetails →
DeepSeek R1 Distill Qwen 7BQ4_K - Medium51.9598LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium48.7584LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium48.7584LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium48.3580LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium47.7562LocalScore ↗4,352llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium47.5553LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium47.2551LocalScore ↗512+128llamafile 0.9.2Details →
Llama 2 7BQ8_043.9719llama.cpp (Metal) ↗512+128llama.cpp MetalDetails →
Meta Llama 3.1 8B InstructQ4_K - Medium42.2475LocalScore ↗4,352llamafile 0.9.2Details →
gemma-2-9b-itQ4_K - Medium36.3418LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium33.7432LocalScore ↗4,352llamafile 0.9.2Details →
DeepSeek R1 Distill Qwen 14BQ4_K - Small27.8293LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium27.3280LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium26.9300LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium26.9300LocalScore ↗4,352llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium26.8301LocalScore ↗4,352llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium26.7309LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium26.4277LocalScore ↗512+128llamafile 0.9.2Details →
Llama 2 7BF1624.3736llama.cpp (Metal) ↗512+128llama.cpp MetalDetails →
Qwen2.5 14B InstructQ4_K - Medium22.4235LocalScore ↗4,352llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium22.3248LocalScore ↗512+128llamafile 0.9.2Details →
Mistral Small 24B Instruct 2501Q4_K - Small21.3185LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium20.2262LocalScore ↗512+128llamafile 0.9.2Details →

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.