Hardware /

Apple M3 (10-core GPU)

26measured results
1,564best gen tok/s*
24 GBreported memory

*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →

llama.cpp

ModelQuantGen tok/sPrompt tok/sSourceContext testedBuild
Qwen2.5 14B InstructQ4_K - Medium1,5649,787LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium68.2944LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium67.9939LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium67.8937LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium67.8938LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium67.0927LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium66.8938LocalScore ↗4,352llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium66.3931LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium65.8906LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium65.6927LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium65.0907LocalScore ↗4,352llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium63.7917LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium59.5874LocalScore ↗512+128llamafile 0.9.2Details →
Llama 2 7BQ4_021.3187llama.cpp (Metal) ↗512+128llama.cpp MetalDetails →
Meta Llama 3.1 8B InstructQ4_K - Medium15.6157LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium13.7137LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium13.6137LocalScore ↗512+128llamafile 0.9.2Details →
Llama 2 7BQ8_012.3188llama.cpp (Metal) ↗512+128llama.cpp MetalDetails →
Meta Llama 3.1 8B InstructQ4_K - Medium11.1123LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium10.8121LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium9.798.5LocalScore ↗512+128llamafile 0.9.3Details →
Models Mlx Community Meta Llama 3.1 8B Instruct Bf16Q6_K7.9113LocalScore ↗512+128llamafile 0.9.3Details →
Qwen2.5 14B InstructQ4_K - Medium6.367.3LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium6.165.2LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium6.064.9LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium4.345.8LocalScore ↗512+128llamafile 0.9.2Details →

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.