Hardware /

GTX 1660 SUPER

16measured results
63.9best gen tok/s*
6 GBreported memory

*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →

llama.cpp

ModelQuantGen tok/sPrompt tok/sSourceContext testedBuild
Qwen2.5 0.5B InstructQ4_K - Medium63.91,507LocalScore ↗512+128llamafile 0.9.3Details →
Qwen2.5 0.5B InstructQ4_K - Medium63.91,516LocalScore ↗512+128llamafile 0.9.3Details →
Qwen2.5 0.5B InstructQ4_K - Medium63.81,520LocalScore ↗512+128llamafile 0.9.3Details →
Qwen1.5-0.5B-Chat-AWQ-fp16Q8_063.11,795LocalScore ↗512+128llamafile 0.9.3Details →
Llama 3.2 1B InstructQ4_K - Medium36.5825LocalScore ↗512+128llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium35.5793LocalScore ↗4,352llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium35.3793LocalScore ↗4,352llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium35.3801LocalScore ↗4,352llamafile 0.9.2Details →
Llama 3.2 1B InstructQ4_K - Medium35.2789LocalScore ↗4,352llamafile 0.9.2Details →
tinyllama_tinyllama-1.1b-chat-v1.0Q4_K - Medium28.5766LocalScore ↗512+128llamafile 0.9.3Details →
tinyllama_tinyllama-1.1b-chat-v1.0Q4_K - Medium28.5764LocalScore ↗512+128llamafile 0.9.3Details →
Qwen2.5 Coder 7B InstructQ4_K - Medium15.7145LocalScore ↗512+128llamafile 0.9.3Details →
Qwen3 Coder 0.6B 0.9B 7 1bR5 Nobs F16F1614.8317LocalScore ↗512+128llamafile 0.9.3Details →
Llama 3.2 3B InstructQ4_K - Medium14.4293LocalScore ↗512+128llamafile 0.9.3Details →
Meta Llama 3.1 8B InstructQ4_K - Medium10.0133LocalScore ↗512+128llamafile 0.9.2Details →
DeepSeek R1 Distill Qwen 1.5BIQ4_XS - 4.25 bpw7.5164LocalScore ↗512+128llamafile 0.9.3Details →

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.