Models /
h2o-danube3-500m-chat
31results
1devices
0.51Bparams
llama.cpp
31 tiny/test model result(s) (excluded from headlines)
| Hardware | Quant | Gen tok/s | Prompt tok/s | Source | Context tested | Build | |
|---|---|---|---|---|---|---|---|
| RTX 4050 Laptop GPU | Q4_K - Medium | 219 | 10,670 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q2_K - Medium | 211 | 9,347 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q2_K - Small | 210 | 8,760 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q4_1 | 206 | 10,482 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q5_0 | 204 | 10,493 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q4_0 | 204 | 10,661 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q3_K - Small | 204 | 9,703 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q4_K - Small | 203 | 10,347 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q5_K - Small | 202 | 10,384 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q3_K - Medium | 201 | 10,136 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q5_K - Medium | 200 | 10,035 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q4_K - Medium | 198 | 10,530 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q3_K - Large | 196 | 10,154 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q5_1 | 196 | 10,424 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q6_K | 189 | 9,569 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | unknown, may not work | 175 | 10,676 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q8_0 | 173 | 9,982 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | F16 | 136 | 9,907 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | all F32 | 80.3 | 6,559 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | BF16 | 47.8 | 5,258 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ1_S - 1.5625 bpw | 38.6 | 150 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ2_XS - 2.3125 bpw | 34.9 | 143 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ2_M - 2.7 bpw | 30.5 | 135 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ1_M - 1.75 bpw | 30.4 | 94.5 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ3_S mix - 3.66 bpw | 30.3 | 122 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ2_S - 2.5 bpw | 30.1 | 131 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ2_XXS - 2.0625 bpw | 28.9 | 138 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ4_XS - 4.25 bpw | 28.4 | 329 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ3_XXS - 3.0625 bpw | 26.3 | 120 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ4_NL - 4.5 bpw | 26.3 | 259 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | IQ3_S - 3.4375 bpw | 24.9 | 94.5 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
Ollama
No measured Ollama results yet. Submit yours →
MLX
No measured MLX results yet. Submit yours →
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.