Models /
Llama 3.2 1B Instruct
699results
154devices
1.50Bparams
432fastest gen tok/s
llama.cpp
| Hardware | Quant | Gen tok/s | Prompt tok/s | Source | Context tested | Build | |
|---|---|---|---|---|---|---|---|
| RTX 4090 | Q4_K - Medium | 432 | 27,767 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 6000 Ada | Q4_K - Medium | 416 | 27,892 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 6000 Ada | Q4_K - Medium | 406 | 26,670 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 396 | 25,164 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 382 | 25,538 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 377 | 15,607 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell | Q4_K - Medium | 374 | 30,896 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 Ti | Q4_K - Medium | 372 | 15,575 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 368 | 15,252 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 367 | 15,400 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| L40S | Q4_K - Medium | 361 | 24,736 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 361 | 14,974 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 360 | 15,064 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 360 | 27,505 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 D | Q4_K - Medium | 359 | 24,549 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 | Q4_K - Medium | 352 | 17,958 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L40S | Q4_K - Medium | 352 | 24,909 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L40S | Q4_K - Medium | 352 | 24,782 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 350 | 14,749 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 348 | 14,452 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 Ti | Q4_K - Medium | 346 | 15,175 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 Ti | Q4_K - Medium | 343 | 14,610 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 342 | 13,768 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 342 | 17,080 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 342 | 14,411 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 340 | 13,805 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 339 | 14,601 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 PCIe | Q4_K - Medium | 335 | 18,590 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 332 | 14,470 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 331 | 13,990 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 330 | 12,190 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 330 | 14,061 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| RTX 3090 | Q4_K - Medium | 328 | 14,169 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 322 | 12,425 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 321 | 14,233 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 321 | 13,357 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 319 | 13,125 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A6000 | Q4_K - Medium | 315 | 13,191 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 315 | 12,562 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 | Q4_K - Medium | 313 | 11,835 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 | Q4_K - Medium | 310 | 12,253 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 309 | 11,156 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 309 | 15,156 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 308 | 11,982 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| A100 80 GB | Q4_K - Medium | 308 | 15,370 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 308 | 12,344 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 | Q4_K - Medium | 307 | 12,325 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 301 | 12,964 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q6_K | 300 | 13,357 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 Ti | Q4_K - Medium | 297 | 10,074 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4500 | Q4_K - Medium | 288 | 10,515 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4500 | Q4_K - Medium | 285 | 10,328 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| A40 | Q4_K - Medium | 285 | 12,204 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 | Q4_K - Medium | 283 | 12,717 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| A40 | Q4_K - Medium | 283 | 12,109 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 282 | 25,807 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla V100-SXM2-16GB | Q4_K - Medium | 278 | 9,858 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 276 | 12,950 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5090 | Q4_K - Medium | 276 | 27,559 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 275 | 27,681 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 | Q6_K | 273 | 11,451 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla V100-SXM2-32GB | Q4_K - Medium | 272 | 9,924 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 263 | 8,943 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 260 | 8,801 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 Ti | Q4_K - Medium | 260 | 9,011 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 260 | 15,534 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 258 | 8,807 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 258 | 8,757 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 257 | 8,758 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 254 | 8,609 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 254 | 15,705 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro RTX 6000 | Q4_K - Medium | 253 | 9,138 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Quadro RTX 8000 | Q4_K - Medium | 250 | 9,356 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla V100-SXM2-16GB | Q4_K - Medium | 250 | 9,743 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla V100-SXM2-32GB | Q4_K - Medium | 249 | 9,892 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla V100-SXM2-16GB | Q4_K - Medium | 248 | 8,984 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 SUPER | Q4_K - Medium | 247 | 17,479 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 244 | 7,658 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 242 | 7,283 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 Ti | Q4_K - Medium | 242 | 8,708 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 241 | 8,404 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 240 | 16,482 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 240 | 16,447 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 SUPER | Q4_K - Medium | 238 | 13,799 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 238 | 16,091 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 238 | 16,125 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 Ada Generation | Q4_K - Medium | 237 | 10,703 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 236 | 9,991 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 235 | 7,901 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 235 | 11,682 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3080 | Q4_K - Medium | 233 | 11,173 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX A4000 | Q4_K - Medium | 233 | 8,278 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Titan V | Q4_K - Medium | 232 | 7,853 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 231 | 8,209 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Max (32-core GPU) | Q4_K - Medium | 230 | 3,727 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Max (32-core GPU) | Q4_K - Medium | 229 | 3,621 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 228 | 11,716 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3080 | Q4_K - Medium | 226 | 11,437 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 226 | 9,965 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 | Q4_K - Medium | 222 | 6,743 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 220 | 9,950 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 219 | 9,424 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 SUPER | Q4_K - Medium | 218 | 13,459 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Quadro RTX 8000 | Q4_K - Medium | 218 | 8,204 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 218 | 9,337 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| RTX 3070 | Q4_K - Medium | 218 | 8,237 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 218 | 9,134 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 216 | 9,463 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 216 | 12,560 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3070 | Q4_K - Medium | 216 | 8,295 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 216 | 10,380 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 214 | 9,161 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 214 | 6,358 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 214 | 28,446 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX A4000 | Q4_K - Medium | 213 | 7,200 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2070 SUPER | Q4_K - Medium | 212 | 6,350 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2070 SUPER | Q4_K - Medium | 212 | 6,082 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 212 | 12,769 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 | Q4_K - Medium | 211 | 6,479 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 211 | 6,061 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 210 | 6,386 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 209 | 11,348 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 208 | 12,930 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 4090 Laptop GPU | Q4_K - Medium | 207 | 11,965 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 206 | 6,119 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 205 | 11,930 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 204 | 8,600 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 2080 Ti | Q4_K - Medium | 202 | 8,883 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 202 | 5,800 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 Laptop GPU | Q4_K - Medium | 202 | 7,018 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 201 | 6,022 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 201 | 16,292 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 200 | 15,093 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 199 | 7,575 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 199 | 14,206 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 198 | 11,534 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3060 | Q4_K - Medium | 197 | 6,058 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2070 | Q4_K - Medium | 196 | 5,665 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Laptop GPU | Q4_K - Medium | 196 | 8,034 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 195 | 7,724 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 2060 SUPER | Q4_K - Medium | 195 | 5,553 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 194 | 13,974 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 193 | 14,107 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Laptop GPU | Q4_K - Medium | 192 | 6,950 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 191 | 3,879 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 SFF Ada Generation | Q4_K - Medium | 191 | 8,762 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 191 | 9,159 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 | Q4_K - Medium | 190 | 7,201 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L4 | Q4_K - Medium | 190 | 8,579 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 190 | 3,888 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 190 | 3,952 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 190 | 8,921 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 SFF Ada Generation | Q4_K - Medium | 189 | 8,786 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L4 | Q4_K - Medium | 189 | 9,573 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2070 SUPER | Q4_K - Medium | 189 | 5,696 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 189 | 14,676 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 | Q4_K - Medium | 188 | 6,899 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| L4 | Q4_K - Medium | 188 | 9,571 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 SFF Ada Generation | Q4_K - Medium | 188 | 8,712 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 186 | 3,699 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 SUPER | Q4_K - Medium | 186 | 7,096 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 186 | 13,196 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti Laptop GPU | Q4_K - Medium | 186 | 11,428 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| L4 | Q4_K - Medium | 185 | 9,351 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 185 | 3,883 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 | Q4_K - Medium | 185 | 7,818 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 185 | 11,628 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 Ti | Q4_K - Medium | 185 | 8,285 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 184 | 3,893 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| RTX 2070 | Q4_K - Medium | 184 | 5,967 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 184 | 3,776 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 Ada Generation | Q4_K - Medium | 183 | 10,193 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 183 | 3,878 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 Laptop GPU | Q4_K - Medium | 182 | 11,598 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 182 | 8,050 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 182 | 3,746 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 181 | 5,836 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 181 | 4,072 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 181 | 3,877 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3500 Ada Generation Laptop GPU | Q4_K - Medium | 181 | 8,206 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 181 | 3,925 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 180 | 3,769 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 180 | 8,826 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 180 | 3,827 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 180 | 3,751 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 180 | 3,918 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 180 | 3,858 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 180 | 3,608 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 179 | 3,814 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 179 | 3,654 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 179 | 3,799 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 179 | 5,483 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 179 | 3,731 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 179 | 3,756 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 179 | 3,896 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 178 | 5,719 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 178 | 8,269 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 178 | 3,658 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 178 | 5,000 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 177 | 3,800 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Ultra (60-core GPU) | Q4_K - Medium | 176 | 3,296 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 176 | 8,411 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 176 | 12,960 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 176 | 8,835 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M5 Pro (20-core GPU) | Q4_K - Medium | 176 | 2,423 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (20-core GPU) | Q4_K - Medium | 175 | 2,403 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (20-core GPU) | Q4_K - Medium | 174 | 2,410 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 174 | 8,575 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M2 Ultra (60-core GPU) | Q4_K - Medium | 174 | 3,290 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 173 | 7,408 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2000 Ada | Q4_K - Medium | 173 | 6,151 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti Laptop GPU | Q4_K - Medium | 173 | 11,833 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti Laptop GPU | Q4_K - Medium | 172 | 11,878 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 171 | 3,301 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 171 | 8,699 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Ultra (60-core GPU) | Q4_K - Medium | 170 | 3,272 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 169 | 7,163 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 169 | 3,211 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 169 | 6,019 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2060 | Q4_K - Medium | 168 | 4,875 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2000 Ada | Q4_K - Medium | 168 | 6,085 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 167 | 3,325 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 165 | 3,316 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 165 | 3,347 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 9070 XT | Q4_K - Medium | 165 | 2,750 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 165 | 3,234 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 2000 Ada | Q4_K - Medium | 165 | 6,025 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 165 | 3,112 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 164 | 3,224 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 164 | 22,081 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 162 | 6,037 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 162 | 3,197 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 162 | 10,791 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 161 | 10,794 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 160 | 6,037 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 160 | 5,174 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 9070 XT | Q4_K - Medium | 160 | 2,665 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Titan Xp | Q4_K - Medium | 160 | 4,298 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 159 | 21,826 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 158 | 7,964 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 158 | 7,437 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 158 | 8,014 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 Laptop GPU | Q4_K - Medium | 157 | 7,533 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 | Q4_K - Medium | 157 | 6,254 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 157 | 5,787 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX A2000 12GB | Q4_K - Medium | 155 | 4,809 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (64-core GPU) | Q4_K - Medium | 154 | 2,998 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 153 | 6,016 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| GTX 1080 Ti | Q4_K - Medium | 153 | 4,259 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DGX Spark | Q4_K - Medium | 153 | 7,578 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 153 | 3,348 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M2 Max (38-core GPU) | Q4_K - Medium | 153 | 2,551 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 153 | 7,662 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 152 | 6,752 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 152 | 4,587 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 151 | 11,382 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 151 | 4,403 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 150 | 3,395 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 148 | 3,403 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (64-core GPU) | Q4_K - Medium | 148 | 2,956 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (38-core GPU) | Q4_K - Medium | 148 | 2,494 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 148 | 3,209 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 148 | 5,276 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 146 | 3,292 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 Ti | Q4_K - Medium | 146 | 4,068 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 Ti | Q4_K - Medium | 146 | 4,075 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 Ti | Q4_K - Medium | 145 | 4,042 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX A2000 12GB | Q4_K - Medium | 144 | 4,363 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 144 | 14,611 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 Ti | Q4_K - Medium | 144 | 4,047 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 144 | 19,804 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3050 Ti Laptop GPU | Q4_K - Medium | 142 | 4,094 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 SUPER | Q4_K - Medium | 141 | 16,736 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (48-core GPU) | Q4_K - Medium | 139 | 2,624 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 139 | 7,083 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 SUPER | Q4_K - Medium | 138 | 12,894 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (48-core GPU) | Q4_K - Medium | 138 | 2,537 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (48-core GPU) | Q4_K - Medium | 137 | 2,586 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 137 | 13,908 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 SUPER | Q4_K - Medium | 137 | 12,969 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 136 | 4,902 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 136 | 13,164 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 135 | 7,232 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M2 Max (30-core GPU) | Q4_K - Medium | 135 | 2,105 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 135 | 6,980 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 135 | 4,908 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 134 | 5,936 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 133 | 13,977 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 133 | 4,439 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (30-core GPU) | Q4_K - Medium | 133 | 2,553 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (30-core GPU) | Q4_K - Medium | 133 | 2,520 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4050 Laptop GPU | Q4_K - Medium | 132 | 5,332 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4050 Laptop GPU | Q4_K - Medium | 132 | 5,231 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Radeon RX 7900 XT | Q4_K - Medium | 132 | 4,185 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Radeon RX 7900 XT | Q4_K - Medium | 131 | 4,184 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 2000 Blackwell Generation Laptop GPU | Q4_K - Medium | 131 | 5,884 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 6000 Ada | Q4_K - Medium | 131 | 19,620 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 | Q4_K - Medium | 130 | 11,035 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (30-core GPU) | Q4_K - Medium | 129 | 2,038 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 129 | 10,817 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 129 | 11,111 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 129 | 10,869 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 129 | 10,847 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 129 | 5,750 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2060 | Q4_K - Medium | 128 | 4,640 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 128 | 2,122 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 128 | 18,389 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 128 | 7,374 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 126 | 5,096 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 126 | 1,971 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 126 | 2,065 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 126 | 2,043 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3050 Ti Laptop GPU | Q4_K - Medium | 126 | 4,078 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2060 | Q4_K - Medium | 125 | 4,618 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2060 | Q4_K - Medium | 125 | 3,784 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 125 | 2,012 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 124 | 2,011 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 124 | 2,060 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 124 | 2,022 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 124 | 1,940 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 Laptop GPU | Q4_K - Medium | 124 | 4,930 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Tesla P100-PCIE-16GB | Q4_K - Medium | 124 | 2,516 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 123 | 2,242 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 123 | 2,151 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 123 | 1,961 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (30-core GPU) | Q4_K - Medium | 123 | 2,049 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A1000 Laptop GPU | Q4_K - Medium | 123 | 3,665 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 123 | 14,483 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (30-core GPU) | Q4_K - Medium | 123 | 1,935 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3050 Ti Laptop GPU | Q4_K - Medium | 122 | 4,089 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 122 | 2,134 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 122 | 17,723 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 122 | 18,272 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L40S | Q4_K - Medium | 121 | 18,255 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 121 | 2,123 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 121 | 2,137 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 7800 XT | Q4_K - Medium | 121 | 2,512 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 120 | 2,150 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 12GB | Q4_K - Medium | 120 | 4,474 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 120 | 2,115 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 119 | 2,126 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 119 | 2,211 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 119 | 2,197 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 119 | 2,062 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 118 | 7,480 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 118 | 2,068 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 118 | 2,164 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L40 | Q4_K - Medium | 118 | 17,686 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 118 | 13,485 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 118 | 2,116 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 XT | Q4_K - Medium | 118 | 4,655 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 118 | 2,135 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 118 | 7,455 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 118 | 2,128 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla P40 | Q4_K - Medium | 118 | 2,965 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 118 | 2,119 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 117 | 2,163 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 117 | 2,108 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 117 | 2,133 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7800 XT | Q4_K - Medium | 117 | 2,778 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| GTX 1070 Ti | Q4_K - Medium | 117 | 3,003 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 | Q4_K - Medium | 117 | 3,811 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| GTX 1070 Ti | Q4_K - Medium | 116 | 2,993 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 116 | 2,071 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 115 | 2,109 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla P100-PCIE-16GB | Q4_K - Medium | 115 | 2,244 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 115 | 11,478 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 114 | 10,350 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| GTX 1070 Ti | Q4_K - Medium | 114 | 2,944 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell | Q4_K - Medium | 114 | 19,907 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 114 | 11,568 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4050 Laptop GPU | Q4_K - Medium | 114 | 4,979 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q4_K - Medium | 113 | 5,176 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 112 | 7,529 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7800 XT | Q6_K | 112 | 3,009 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 112 | 1,859 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 112 | 12,246 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 111 | 1,889 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 111 | 1,774 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 111 | 1,782 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 111 | 1,735 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 111 | 1,846 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 111 | 1,736 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 110 | 1,780 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 110 | 1,845 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 110 | 1,806 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q5_K - Medium | 110 | 5,591 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 110 | 10,769 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6800 XT | Q4_K - Medium | 110 | 4,585 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 110 | 17,936 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6650 XT | Q4_K - Medium | 109 | 3,031 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| RTX A2000 8GB Laptop GPU | Q4_K - Medium | 109 | 3,399 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q6_K | 109 | 5,147 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | Q4_K - Medium | 109 | 16,148 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 109 | 2,585 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3050 6GB Laptop GPU | Q4_K - Medium | 108 | 3,709 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 7800 XT | Q4_K - Medium | 108 | 3,091 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3050 | Q4_K - Medium | 108 | 3,407 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3050 | Q4_K - Medium | 107 | 3,289 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 107 | 13,136 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 106 | 8,549 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 106 | 1,684 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 106 | 1,644 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 106 | 10,130 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell | Q4_K - Medium | 106 | 18,081 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 Laptop GPU | Q4_K - Medium | 106 | 3,077 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 105 | 11,674 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 105 | 12,483 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 105 | 2,458 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 Laptop GPU | Q4_K - Medium | 105 | 3,091 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7800 XT | Q8_0 | 105 | 3,073 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 105 | 1,680 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 105 | 17,167 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 105 | 1,627 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 105 | 7,346 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | Q4_K - Medium | 104 | 16,219 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (19-core GPU) | Q4_K - Medium | 104 | 1,508 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5080 | Q4_K - Medium | 103 | 13,606 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1070 | Q4_K - Medium | 103 | 2,574 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DGX Spark | Q4_K - Medium | 103 | 5,148 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6700 XT | Q4_K - Medium | 103 | 3,192 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 103 | 8,234 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q8_0 | 103 | 6,338 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 102 | 18,337 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon PRO V620 | Q4_K - Medium | 102 | 4,607 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon PRO V620 | Q4_K - Medium | 102 | 4,609 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 | Q4_K - Medium | 101 | 3,829 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3050 | Q4_K - Medium | 101 | 3,337 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6700 XT | Q4_K - Medium | 101 | 3,088 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 101 | 1,272 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 101 | 1,266 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 5070 | Q4_K - Medium | 101 | 8,328 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 101 | 1,293 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (19-core GPU) | Q4_K - Medium | 100 | 1,473 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 100 | 11,177 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DGX Spark | Q4_K - Medium | 100 | 5,436 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (19-core GPU) | Q4_K - Medium | 99.6 | 1,485 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (19-core GPU) | Q4_K - Medium | 99.5 | 1,458 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (19-core GPU) | Q4_K - Medium | 99.3 | 1,495 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 99.2 | 1,265 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 99.1 | 1,272 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H200 | Q4_K - Medium | 99.0 | 15,261 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H200 | Q4_K - Medium | 98.7 | 15,044 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H200 | Q4_K - Medium | 98.5 | 14,963 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 98.3 | 9,569 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M2 Pro (19-core GPU) | Q4_K - Medium | 98.0 | 1,477 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3050 | Q4_K - Medium | 98.0 | 3,355 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 97.9 | 8,630 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 97.8 | 1,283 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H200 | Q4_K - Medium | 97.8 | 13,517 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Laptop GPU | Q4_K - Medium | 97.3 | 6,259 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 97.1 | 1,217 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1070 | Q4_K - Medium | 97.0 | 2,378 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 80 GB | Q4_K - Medium | 96.6 | 14,632 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 96.2 | 1,250 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 XT | Q4_K - Medium | 96.1 | 4,565 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 80 GB | Q4_K - Medium | 95.8 | 14,584 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 | Q4_K - Medium | 95.6 | 3,575 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Laptop GPU | Q4_K - Medium | 95.6 | 6,191 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro P4000 | Q4_K - Medium | 94.3 | 2,191 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4050 Laptop GPU | Q4_K - Medium | 94.1 | 4,127 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Quadro P4000 | Q4_K - Medium | 93.9 | 2,197 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| A10 | Q4_K - Medium | 93.5 | 8,777 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 92.8 | 1,689 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 92.6 | 1,638 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6700 XT | Q4_K - Medium | 92.5 | 3,054 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 91.9 | 6,509 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 91.5 | 1,341 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 91.5 | 1,281 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 91.0 | 9,623 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 91.0 | 1,347 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 90.9 | 1,305 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 90.8 | 1,316 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3050 6GB Laptop GPU | Q4_K - Medium | 90.6 | 3,044 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 4070 Laptop GPU | Q4_K - Medium | 90.1 | 6,763 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 89.9 | 1,616 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 89.6 | 1,556 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 88.7 | 1,243 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 NVL | Q4_K - Medium | 88.7 | 13,774 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 88.3 | 1,333 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 88.2 | 1,323 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 88.1 | 1,344 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 87.8 | 7,272 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 86.8 | 7,372 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 86.4 | 3,700 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 85.6 | 1,574 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 84.4 | 1,417 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6650 XT | Q4_K - Medium | 84.0 | 2,494 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 NVL | Q4_K - Medium | 83.9 | 13,164 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6650 XT | Q4_K - Medium | 83.9 | 2,606 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 83.0 | 1,537 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6650 XT | Q4_K - Medium | 83.0 | 2,592 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6650 XT | Q4_K - Medium | 82.3 | 2,475 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 82.3 | 1,197 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6650 XT | Q4_K - Medium | 82.2 | 2,600 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| A100 80 GB | Q4_K - Medium | 81.9 | 10,806 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 500 Ada Generation Laptop GPU | Q4_K - Medium | 81.6 | 3,624 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 81.1 | 1,198 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 80.5 | 1,204 | LocalScore ↗ | 512+128 | llamafile 0.9.0 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 80.0 | 1,192 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 79.1 | 1,177 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| P106-100 | Q4_K - Medium | 78.8 | 1,874 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 78.8 | 1,204 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| P106-100 | Q4_K - Medium | 78.7 | 1,874 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 78.7 | 1,150 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| P106-100 | Q4_K - Medium | 78.7 | 1,879 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 78.6 | 1,096 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 78.5 | 1,193 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 78.4 | 1,141 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 NVL | Q4_K - Medium | 78.3 | 13,821 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 78.1 | 1,126 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 78.1 | 1,190 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1060 6GB | Q4_K - Medium | 78.0 | 1,875 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 78.0 | 1,134 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1060 6GB | Q4_K - Medium | 77.9 | 1,885 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 77.7 | 1,104 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 77.2 | 1,086 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 77.1 | 1,111 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 | Q4_K - Medium | 77.0 | 3,504 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 76.9 | 1,097 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 76.6 | 1,110 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 76.6 | 1,105 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 76.6 | 1,101 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 76.4 | 1,103 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 76.0 | 1,114 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 75.9 | 1,116 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 75.6 | 1,122 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 | Q4_K - Medium | 75.4 | 3,476 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 75.2 | 1,100 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 75.2 | 1,113 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1060 6GB | Q4_K - Medium | 74.9 | 1,828 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| GTX 1080 | Q4_K - Medium | 74.8 | 3,431 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 74.6 | 1,009 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 74.5 | 1,064 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 74.4 | 982 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 74.0 | 1,020 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 74.0 | 992 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| P106-100 | Q4_K - Medium | 73.9 | 1,812 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| P106-100 | Q4_K - Medium | 73.9 | 1,823 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 73.4 | 1,008 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6600 | Q4_K - Medium | 72.9 | 2,052 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 72.9 | 1,070 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 72.7 | 1,065 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon Graphics | Q4_K - Medium | 72.6 | 599 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4500 Laptop GPU | Q4_K - Medium | 72.5 | 3,868 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 72.5 | 950 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 72.5 | 1,078 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 72.5 | 1,061 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 72.3 | 1,047 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 71.0 | 1,064 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 70.8 | 5,472 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 69.2 | 964 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 68.6 | 1,068 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 68.2 | 944 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 67.9 | 939 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 67.8 | 937 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 67.8 | 938 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 67.0 | 927 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q5_K - Medium | 66.8 | 1,079 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 66.8 | 938 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| GTX 1060 6GB | Q8_0 | 66.8 | 1,921 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 66.3 | 855 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 66.3 | 931 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (8-core GPU) | Q4_K - Medium | 66.3 | 904 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (8-core GPU) | Q4_K - Medium | 66.3 | 902 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (8-core GPU) | Q4_K - Medium | 66.2 | 895 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 66.1 | 12,782 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (8-core GPU) | Q4_K - Medium | 65.8 | 903 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 65.8 | 906 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 65.6 | 927 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (8-core GPU) | Q4_K - Medium | 65.6 | 898 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (8-core GPU) | Q4_K - Medium | 65.5 | 886 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 65.0 | 907 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 63.7 | 917 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla P40 | Q4_K - Medium | 63.4 | 1,452 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L40S-8Q | Q4_K - Medium | 62.8 | 12,461 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| L40S-8Q | Q4_K - Medium | 62.1 | 12,666 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M3 (8-core GPU) | Q4_K - Medium | 60.3 | 804 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 60.0 | 837 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 60.0 | 835 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 60.0 | 846 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 59.9 | 827 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 59.5 | 874 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro P2000 | Q4_K - Medium | 58.3 | 1,353 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 57.7 | 793 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7600 XT | Q4_K - Medium | 57.1 | 1,502 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 57.0 | 831 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 57.0 | 819 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 56.5 | 805 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Radeon RX 5500 XT | Q4_K - Medium | 56.2 | 174 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6900 XT | F16 | 55.5 | 3,235 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 55.0 | 8,761 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| P106-100 | Q4_K - Medium | 54.8 | 1,525 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 53.9 | 822 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 53.4 | 9,413 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| P106-100 | Q4_K - Medium | 53.3 | 1,520 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (8-core GPU) | Q4_K - Medium | 52.9 | 710 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (8-core GPU) | Q4_K - Medium | 52.8 | 717 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 52.6 | 6,655 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 52.5 | 6,597 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 48.3 | 774 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2050 | Q4_K - Medium | 47.9 | 1,770 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Orin | Q4_K - Medium | 47.8 | 1,233 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 47.7 | 9,439 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| GTX 1080 | Q4_K - Medium | 46.9 | 1,057 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 46.8 | 9,444 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| GTX 1050 Ti | Q4_K - Medium | 46.7 | 1,053 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 45.7 | 11,191 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5090 | Q4_K - Medium | 45.3 | 10,888 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 45.3 | 8,635 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 5090 | Q4_K - Medium | 45.1 | 10,856 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5090 | Q4_K - Medium | 45.0 | 11,332 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| GTX 1050 Ti | Q4_K - Medium | 44.9 | 1,002 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| GTX 1050 Ti | Q4_K - Medium | 44.7 | 986 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| GTX 1050 Ti | Q4_K - Medium | 44.5 | 1,013 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 44.0 | 8,711 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M2 (8-core GPU) | Q5_K - Medium | 43.5 | 662 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5090 | Q4_K - Medium | 43.5 | 10,900 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4090 | Q4_K - Medium | 43.4 | 10,569 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 43.1 | 725 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Orin | Q4_K - Medium | 42.7 | 1,082 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| P106-090 | Q4_K - Medium | 42.1 | 958 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5080 | Q4_K - Medium | 41.7 | 9,611 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| GTX 1050 | Q4_K - Medium | 41.5 | 959 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 41.5 | 10,392 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 41.4 | 612 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 41.4 | 9,013 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 41.4 | 10,144 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 41.2 | 608 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5080 | Q4_K - Medium | 41.2 | 9,533 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5080 | Q4_K - Medium | 41.1 | 9,524 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5080 | Q4_K - Medium | 41.0 | 9,253 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 40.9 | 601 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 40.7 | 615 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 40.6 | 579 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 39.7 | 606 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (8-core GPU) | Q4_K - Medium | 39.1 | 632 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 39.1 | 596 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 39.0 | 578 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 38.9 | 537 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 38.7 | 579 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 38.6 | 536 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 38.5 | 544 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 38.4 | 540 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 38.4 | 528 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 38.3 | 535 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 37.3 | 509 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| GTX 1660 SUPER | Q4_K - Medium | 36.5 | 825 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| P106-090 | Q4_K - Medium | 36.3 | 887 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 36.0 | 503 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1660 SUPER | Q4_K - Medium | 35.5 | 793 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| GTX 1660 SUPER | Q4_K - Medium | 35.3 | 793 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| GTX 1660 SUPER | Q4_K - Medium | 35.3 | 801 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| GTX 1660 SUPER | Q4_K - Medium | 35.2 | 789 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| GTX 1050 | Q4_K - Medium | 35.0 | 792 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5080 Laptop GPU | Q4_K - Medium | 34.7 | 5,678 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Radeon RX 7900 XT | Q4_K - Medium | 34.6 | 6,502 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple A18 Pro 2P+4E+5GPU | Q4_K - Medium | 34.0 | 433 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1650 SUPER | Q4_K - Medium | 31.2 | 667 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| GTX 1050 | Q4_K - Medium | 27.3 | 692 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro P620 | Q4_K - Medium | 26.6 | 613 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Quadro P620 | Q4_K - Medium | 26.4 | 611 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M2 (8-core GPU) | Q4_K - Medium | 24.6 | 429 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1070 | Q4_K - Medium | 24.4 | 814 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GT 1030 | Q4_K - Medium | 24.1 | 543 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1650 | Q4_K - Medium | 23.8 | 521 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4090 | Q4_K - Medium | 22.3 | 6,976 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 22.1 | 369 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1650 | Q4_K - Medium | 21.8 | 490 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M2 (8-core GPU) | Q4_K - Medium | 21.3 | 322 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| T1200 Laptop GPU | Q4_K - Medium | 21.0 | 540 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1650 Ti | Q4_K - Medium | 20.7 | 567 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1650 | Q4_K - Medium | 20.6 | 568 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| GTX 1650 Ti | Q4_K - Medium | 19.5 | 547 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Quadro P620 | Q4_K - Medium | 18.6 | 330 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M2 (8-core GPU) | Q4_K - Medium | 17.8 | 330 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7900 XTX | Q4_0 | 16.6 | 3,681 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 16.4 | 3,644 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon Graphics | Q4_K - Medium | 16.2 | 1,646 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 16.0 | 3,734 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7800 XT | Q4_0 | 13.9 | 2,435 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| MX150 | Q4_K - Medium | 13.5 | 311 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1050 | Q4_K - Medium | 13.3 | 118 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6700 XT | Q4_K - Medium | 12.8 | 2,552 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6700 XT | Q4_K - Medium | 12.4 | 2,532 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q4_K - Medium | 12.3 | 3,646 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 XT | Q4_K - Medium | 12.1 | 3,324 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 12.1 | 632 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 11.8 | 666 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 11.7 | 690 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6600 | Q4_K - Medium | 11.5 | 1,908 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 11.5 | 617 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 11.2 | 572 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon Graphics | Q4_K - Medium | 10.7 | 565 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon 760M Graphics | Q4_K - Medium | 10.4 | 480 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6800 | Q4_K - Medium | 10.0 | 2,563 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 7.5 | 506 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon 740M Graphics | Q4_K - Medium | 6.2 | 212 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 5.9 | 399 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
Ollama
No measured Ollama results yet. Submit yours →
MLX
No measured MLX results yet. Submit yours →
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.