Hardware /

AMD Radeon RX 6900 XT

30measured results
113best gen tok/s*
16 GBreported memory

*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →

llama.cpp

ModelQuantGen tok/sPrompt tok/sSourceContext testedBuild
Llama 3.2 1B InstructQ4_K - Medium1135,176LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ5_K - Medium1105,591LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ6_K1095,147LocalScore ↗512+128llamafile 0.9.2Details →
Llama 2 7BQ4_01081,901llama.cpp (Vulkan) ↗512+128llama.cpp Vulkan @a972faeDetails →
Llama 2 7BQ4_01061,762llama.cpp (Vulkan) ↗512+128llama.cpp Vulkan @a972faeDetails →
Llama 3.2 1B InstructQ8_01036,338LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructF1655.53,235LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 3B InstructQ4_K - Medium55.32,159LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 3B InstructQ5_K - Medium53.62,339LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 3B InstructQ6_K52.82,124LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium50.31,170LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 3B InstructQ8_049.12,680LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ6_K44.51,201LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium40.31,028LocalScore ↗512+128llamafile 0.9.3Details →
Meta Llama 3.1 8B InstructQ8_038.31,600LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium36.71,050LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ5_K - Medium34.91,157LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ6_K33.11,016LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ8_029.51,389LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 3B InstructF1629.11,475LocalScore ↗512+128llamafile 0.9.2Details →
Mistral Nemo Instruct 2407Q4_K - Medium27.1746LocalScore ↗512+128llamafile 0.9.2Details →
Mistral Nemo Instruct 2407Q5_K - Medium25.4717LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium24.5612LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium24.2574LocalScore ↗512+128llamafile 0.9.2Details →
Mistral Nemo Instruct 2407Q6_K23.8647LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium22.2593LocalScore ↗512+128llamafile 0.9.2Details →
Mistral Nemo Instruct 2407Q8_020.7832LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructF1617.71,243LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium12.33,646LocalScore ↗512+128llamafile 0.9.2Details →
Qwen2.5 14B InstructQ4_K - Medium3.0496LocalScore ↗512+128llamafile 0.9.2Details →

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.