Llama 3.2 1B Instruct on RTX 5090

360best gen tok/s

28,446best prompt tok/s

640max context tested

19results

Measured · all numbers below come from the linked sources

llama.cpp

Quant	Gen tok/s	Prompt tok/s	Source	Context tested	TTFT	Build
Q4_K - Medium	360	27,505	LocalScore ↗	512+128	48 ms	llamafile 0.9.2
Q4_K - Medium	276	27,559	LocalScore ↗	512+128	2,135 ms	llamafile 0.9.2
Q4_K - Medium	275	27,681	LocalScore ↗	512+128	56 ms	llamafile 0.9.2
Q4_K - Medium	214	28,446	LocalScore ↗	512+128	48 ms	llamafile 0.9.3
Q4_K - Medium	159	21,826	LocalScore ↗	512+128	1,699 ms	llamafile 0.9.2
Q4_K - Medium	128	18,389	LocalScore ↗	512+128	71 ms	llamafile 0.9.2
Q4_K - Medium	122	17,723	LocalScore ↗	512+128	73 ms	llamafile 0.9.2
Q4_K - Medium	122	18,272	LocalScore ↗	512+128	71 ms	llamafile 0.9.2
Q4_K - Medium	110	17,936	LocalScore ↗	512+128	73 ms	llamafile 0.9.2
Q4_K - Medium	105	12,483	LocalScore ↗	512+128	325 ms	llamafile 0.9.2
Q4_K - Medium	105	17,167	LocalScore ↗	512+128	75 ms	llamafile 0.9.2
Q4_K - Medium	102	18,337	LocalScore ↗	512+128	73 ms	llamafile 0.9.2
Q4_K - Medium	45.7	11,191	LocalScore ↗	512+128	123 ms	llamafile 0.9.3
Q4_K - Medium	45.3	10,888	LocalScore ↗	512+128	128 ms	llamafile 0.9.3
Q4_K - Medium	45.1	10,856	LocalScore ↗	512+128	127 ms	llamafile 0.9.3
Q4_K - Medium	45.0	11,332	LocalScore ↗	512+128	122 ms	llamafile 0.9.3
Q4_K - Medium	43.5	10,900	LocalScore ↗	512+128	127 ms	llamafile 0.9.3
Q4_K - Medium	41.5	10,392	LocalScore ↗	512+128	135 ms	llamafile 0.9.3
Q4_K - Medium	41.4	10,144	LocalScore ↗	512+128	136 ms	llamafile 0.9.3

Ollama

No measured Ollama results yet. Submit yours →

MLX

No measured MLX results yet. Submit yours →

"Context tested" is the largest prompt+generation length actually benchmarked (tokens), not a memory-fit estimate.

What else can RTX 5090 run? →

Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.