RTX 4050 Laptop GPU × h2o-danube3-500m-chat
h2o-danube3-500m-chat on RTX 4050 Laptop GPU
219best gen tok/s
10,676best prompt tok/s
640max context tested
31results
Measured · all numbers below come from the linked sources
llama.cpp
31 tiny/test model result(s) (excluded from headlines)
| Quant | Gen tok/s | Prompt tok/s | Source | Context tested | TTFT | Build |
|---|---|---|---|---|---|---|
| Q4_K - Medium | 219 | 10,670 | LocalScore ↗ | 512+128 | 141 ms | llamafile 0.9.3 |
| Q2_K - Medium | 211 | 9,347 | LocalScore ↗ | 512+128 | 159 ms | llamafile 0.9.3 |
| Q2_K - Small | 210 | 8,760 | LocalScore ↗ | 512+128 | 162 ms | llamafile 0.9.3 |
| Q4_1 | 206 | 10,482 | LocalScore ↗ | 512+128 | 145 ms | llamafile 0.9.3 |
| Q5_0 | 204 | 10,493 | LocalScore ↗ | 512+128 | 143 ms | llamafile 0.9.3 |
| Q4_0 | 204 | 10,661 | LocalScore ↗ | 512+128 | 144 ms | llamafile 0.9.3 |
| Q3_K - Small | 204 | 9,703 | LocalScore ↗ | 512+128 | 150 ms | llamafile 0.9.3 |
| Q4_K - Small | 203 | 10,347 | LocalScore ↗ | 512+128 | 145 ms | llamafile 0.9.3 |
| Q5_K - Small | 202 | 10,384 | LocalScore ↗ | 512+128 | 145 ms | llamafile 0.9.3 |
| Q3_K - Medium | 201 | 10,136 | LocalScore ↗ | 512+128 | 147 ms | llamafile 0.9.3 |
| Q5_K - Medium | 200 | 10,035 | LocalScore ↗ | 512+128 | 148 ms | llamafile 0.9.3 |
| Q4_K - Medium | 198 | 10,530 | LocalScore ↗ | 512+128 | 144 ms | llamafile 0.9.3 |
| Q3_K - Large | 196 | 10,154 | LocalScore ↗ | 512+128 | 147 ms | llamafile 0.9.3 |
| Q5_1 | 196 | 10,424 | LocalScore ↗ | 512+128 | 144 ms | llamafile 0.9.3 |
| Q6_K | 189 | 9,569 | LocalScore ↗ | 512+128 | 154 ms | llamafile 0.9.3 |
| unknown, may not work | 175 | 10,676 | LocalScore ↗ | 512+128 | 143 ms | llamafile 0.9.3 |
| Q8_0 | 173 | 9,982 | LocalScore ↗ | 512+128 | 147 ms | llamafile 0.9.3 |
| F16 | 136 | 9,907 | LocalScore ↗ | 512+128 | 147 ms | llamafile 0.9.3 |
| all F32 | 80.3 | 6,559 | LocalScore ↗ | 512+128 | 210 ms | llamafile 0.9.3 |
| BF16 | 47.8 | 5,258 | LocalScore ↗ | 512+128 | 255 ms | llamafile 0.9.3 |
| IQ1_S - 1.5625 bpw | 38.6 | 150 | LocalScore ↗ | 512+128 | 8,368 ms | llamafile 0.9.3 |
| IQ2_XS - 2.3125 bpw | 34.9 | 143 | LocalScore ↗ | 512+128 | 8,829 ms | llamafile 0.9.3 |
| IQ2_M - 2.7 bpw | 30.5 | 135 | LocalScore ↗ | 512+128 | 9,293 ms | llamafile 0.9.3 |
| IQ1_M - 1.75 bpw | 30.4 | 94.5 | LocalScore ↗ | 512+128 | 14,702 ms | llamafile 0.9.3 |
| IQ3_S mix - 3.66 bpw | 30.3 | 122 | LocalScore ↗ | 512+128 | 10,696 ms | llamafile 0.9.3 |
| IQ2_S - 2.5 bpw | 30.1 | 131 | LocalScore ↗ | 512+128 | 10,290 ms | llamafile 0.9.3 |
| IQ2_XXS - 2.0625 bpw | 28.9 | 138 | LocalScore ↗ | 512+128 | 9,525 ms | llamafile 0.9.3 |
| IQ4_XS - 4.25 bpw | 28.4 | 329 | LocalScore ↗ | 512+128 | 3,709 ms | llamafile 0.9.3 |
| IQ3_XXS - 3.0625 bpw | 26.3 | 120 | LocalScore ↗ | 512+128 | 10,942 ms | llamafile 0.9.3 |
| IQ4_NL - 4.5 bpw | 26.3 | 259 | LocalScore ↗ | 512+128 | 4,865 ms | llamafile 0.9.3 |
| IQ3_S - 3.4375 bpw | 24.9 | 94.5 | LocalScore ↗ | 512+128 | 14,041 ms | llamafile 0.9.3 |
Ollama
No measured Ollama results yet. Submit yours →
MLX
No measured MLX results yet. Submit yours →
"Context tested" is the largest prompt+generation length actually benchmarked (tokens), not a memory-fit estimate.
What else can RTX 4050 Laptop GPU run? →
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.