Hardware /

RTX 3050

13measured results
108best gen tok/s*
6 GBreported memory

*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →

llama.cpp

ModelQuantGen tok/sPrompt tok/sSourceContext testedBuild
Llama 3.2 1B InstructQ4_K - Medium1083,407LocalScore ↗4,352llamafile 0.9.3Details →
Llama 3.2 1B InstructQ4_K - Medium1073,289LocalScore ↗4,352llamafile 0.9.3Details →
Llama 3.2 1B InstructQ4_K - Medium1013,337LocalScore ↗4,352llamafile 0.9.3Details →
Llama 3.2 1B InstructQ4_K - Medium98.03,355LocalScore ↗512+128llamafile 0.9.2Details →
DeepSeek R1 Distill Qwen 1.5BQ4_K - Medium60.42,851LocalScore ↗512+128llamafile 0.9.2Details →
Gemma 3 4b ItQ4_K - Medium37.01,421LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium35.0965LocalScore ↗4,352llamafile 0.9.3Details →
Meta Llama 3.1 8B InstructQ4_K - Medium34.5979LocalScore ↗512+128llamafile 0.9.2Details →
DeepSeek R1 Distill Qwen 7BQ4_K - Medium27.3911LocalScore ↗512+128llamafile 0.9.2Details →
DeepSeek R1 Distill Llama 8BQ4_K - Medium26.0813LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium26.0760LocalScore ↗512+128llamafile 0.9.2Details →
Meta Llama 3.1 8B InstructQ4_K - Medium25.8797LocalScore ↗4,352llamafile 0.9.3Details →
Granite 3.3 8b InstructQ4_K - Medium23.5594LocalScore ↗512+128llamafile 0.9.2Details →

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.