Hardware /

Apple M1 Max (32-core GPU)

46measured results
128best gen tok/s*
64 GBreported memory

*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →

llama.cpp

ModelQuantGen tok/sPrompt tok/sSourceContext testedBuild
Llama 3.2 1B InstructQ4_K - Medium1282,122LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1261,971LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1262,065LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1262,043LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1252,012LocalScore ↗4,352llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1242,011LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1242,060LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1242,022LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1241,940LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1231,961LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium1101,845LocalScore ↗512+128llamafile 0.9.2Details →
Qwen3 Embedding 0.6bQ8_01102,949LocalScore ↗512+128llamafile 0.9.3Details →
Llama 3.2 1B InstructQ4_K - Medium92.81,689LocalScore ↗4,352llamafile 0.9.2Details →
Llama 2 7BQ4_061.2530llama.cpp (Metal) ↗512+128llama.cpp MetalDetails →
Llama 2 7BQ8_040.2537llama.cpp (Metal) ↗512+128llama.cpp MetalDetails →
Meta Llama 3.1 8B InstructQ4_K - Medium38.4392LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium38.1384LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium37.9385LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium37.7383LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium37.7388LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium37.6389LocalScore ↗4,352llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium37.5369LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium33.6350LocalScore ↗512+128llamafile 0.9.2Details →
Granite 3.2 8b InstructQ4_K - Medium32.6332LocalScore ↗512+128llamafile 0.9.3Details →
Meta Llama 3.1 8B InstructQ4_K - Medium32.3338LocalScore ↗4,352llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium31.6312LocalScore ↗512+128llamafile 0.9.2Details →
Llama 2 7BF1623.0600llama.cpp (Metal) ↗512+128llama.cpp MetalDetails →
Qwen2.5 14B InstructQ4_K - Medium21.4203LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium21.1205LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium21.1205LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium21.0207LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium20.9204LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium20.9199LocalScore ↗4,352llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium20.9203LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium20.7203LocalScore ↗4,352llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium20.7201LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium20.6204LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium20.5191LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium20.4205LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium19.6197LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium18.7211LocalScore ↗4,352llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium18.5188LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium18.1178LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium17.2170LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium16.8162LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium11.9111LocalScore ↗512+128llamafile 0.9.2Details →

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.