Models /
Qwen2.5 14B Instruct
506results
114devices
14.77Bparams
1,564fastest gen tok/s
llama.cpp
| Hardware | Quant | Gen tok/s | Prompt tok/s | Source | Context tested | Build | |
|---|---|---|---|---|---|---|---|
| Apple M3 (10-core GPU) | Q4_K - Medium | 1,564 | 9,787 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell | Q4_K - Medium | 84.8 | 5,568 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell | Q4_K - Medium | 81.1 | 5,126 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell | Q4_K - Medium | 79.7 | 5,072 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 72.5 | 4,361 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 PCIe | Q4_K - Medium | 72.2 | 2,973 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 70.8 | 3,828 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 D | Q4_K - Medium | 70.5 | 3,905 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 6000 Ada | Q4_K - Medium | 70.0 | 3,552 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 69.6 | 4,110 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 68.8 | 4,208 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 68.1 | 2,187 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| A100 80 GB | Q4_K - Medium | 66.3 | 2,069 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 65.6 | 3,809 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 Ti | Q4_K - Medium | 65.5 | 2,320 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 65.3 | 4,777 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 65.1 | 4,787 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 65.1 | 2,185 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 65.0 | 4,758 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 Ti | Q4_K - Medium | 64.6 | 2,317 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 63.7 | 2,135 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 63.6 | 2,151 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 63.4 | 2,131 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 Ti | Q4_K - Medium | 62.4 | 1,969 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 62.3 | 5,328 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 62.0 | 2,126 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 61.9 | 2,003 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 61.8 | 2,106 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 61.6 | 2,083 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 61.0 | 1,917 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 60.6 | 1,911 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 60.5 | 2,034 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 60.3 | 4,106 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 60.2 | 2,014 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 59.9 | 2,005 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 59.9 | 2,090 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 59.8 | 2,030 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 59.8 | 2,010 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 59.7 | 1,837 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 59.7 | 4,122 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 59.6 | 2,049 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 59.4 | 1,944 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 59.1 | 1,876 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 59.0 | 2,007 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 58.9 | 1,881 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 58.6 | 1,878 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 58.4 | 1,873 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 58.4 | 1,938 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 58.3 | 1,854 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 58.2 | 1,821 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4080 SUPER | Q4_K - Medium | 57.4 | 2,938 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 56.7 | 1,877 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 55.6 | 5,069 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 55.5 | 1,884 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 55.2 | 1,728 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 53.6 | 2,526 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 53.2 | 1,882 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 52.3 | 1,719 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 | Q4_K - Medium | 52.1 | 2,665 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 52.1 | 1,705 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 52.0 | 1,595 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A6000 | Q4_K - Medium | 52.0 | 1,883 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 51.8 | 4,028 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tesla V100-SXM2-32GB | Q4_K - Medium | 51.4 | 1,197 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 | Q4_K - Medium | 50.9 | 1,524 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 50.7 | 2,189 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 | Q4_K - Medium | 50.0 | 1,541 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 50.0 | 1,959 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5090 | Q4_K - Medium | 49.5 | 4,295 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 | Q4_K - Medium | 48.9 | 2,778 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 48.7 | 1,957 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tesla V100-SXM2-32GB | Q4_K - Medium | 48.7 | 1,161 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 48.7 | 1,943 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4080 | Q4_K - Medium | 48.2 | 2,747 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 48.1 | 1,916 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 4090 | Q4_K - Medium | 48.0 | 3,965 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3090 | Q4_K - Medium | 48.0 | 1,923 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 47.7 | 2,317 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 47.3 | 2,382 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 46.6 | 1,741 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| A40 | Q4_K - Medium | 46.2 | 1,816 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| A40 | Q4_K - Medium | 46.2 | 1,834 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 SUPER | Q4_K - Medium | 46.1 | 2,803 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5070 | Q4_K - Medium | 45.7 | 1,758 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 44.9 | 1,762 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX A4500 | Q4_K - Medium | 44.1 | 1,332 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4500 | Q4_K - Medium | 43.8 | 1,330 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| TITAN RTX | Q4_K - Medium | 43.3 | 1,224 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro RTX 8000 | Q4_K - Medium | 42.9 | 1,178 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 | Q4_K - Medium | 42.6 | 1,752 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| TITAN RTX | Q4_K - Medium | 42.2 | 1,176 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Titan V | Q4_K - Medium | 42.2 | 962 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 Ti | Q4_K - Medium | 42.2 | 1,166 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 41.8 | 2,174 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 41.2 | 2,490 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3080 | Q4_K - Medium | 41.0 | 1,307 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 40.9 | 1,852 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5090 Laptop GPU | Q4_K - Medium | 40.5 | 1,550 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 40.3 | 2,367 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| A40-48Q | Q4_K - Medium | 39.5 | 1,707 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 39.4 | 2,224 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 39.4 | 1,493 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 2080 Ti | Q4_K - Medium | 39.1 | 1,084 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 38.8 | 2,212 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 SUPER | Q4_K - Medium | 38.8 | 2,012 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Max (40-core GPU) | Q4_K - Medium | 37.5 | 371 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti Laptop GPU | Q4_K - Medium | 37.5 | 1,211 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 2080 Ti | Q4_K - Medium | 37.3 | 1,170 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro RTX 6000 | Q4_K - Medium | 37.1 | 1,077 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 36.8 | 1,425 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 36.7 | 568 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro RTX 8000 | Q4_K - Medium | 36.7 | 1,087 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Ultra (76-core GPU) | Q4_K - Medium | 36.6 | 471 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 36.5 | 1,465 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Ultra (76-core GPU) | Q4_K - Medium | 36.5 | 469 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 36.5 | 2,367 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 36.4 | 1,366 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M5 Max (40-core GPU) | Q4_K - Medium | 36.3 | 338 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Max (40-core GPU) | Q4_K - Medium | 36.2 | 350 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti Laptop GPU | Q4_K - Medium | 36.1 | 1,595 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 36.1 | 2,131 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4070 | Q4_K - Medium | 36.0 | 1,681 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 35.9 | 590 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 35.9 | 579 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 35.9 | 568 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 35.8 | 583 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 35.7 | 580 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 35.0 | 1,410 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 | Q4_K - Medium | 35.0 | 1,642 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 34.8 | 1,086 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (60-core GPU) | Q4_K - Medium | 34.4 | 445 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 34.3 | 1,320 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Max (32-core GPU) | Q4_K - Medium | 34.3 | 343 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Ultra (60-core GPU) | Q4_K - Medium | 34.2 | 381 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 34.2 | 1,410 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 34.1 | 1,100 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 34.0 | 1,429 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Max (32-core GPU) | Q4_K - Medium | 33.8 | 341 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 33.8 | 1,418 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 33.6 | 1,076 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Ultra (60-core GPU) | Q4_K - Medium | 33.6 | 377 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (64-core GPU) | Q4_K - Medium | 33.3 | 377 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 33.2 | 1,062 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 33.2 | 1,044 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 33.2 | 1,052 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 33.1 | 1,057 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 33.1 | 1,050 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 32.9 | 1,040 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 32.9 | 1,045 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 32.9 | 1,040 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 32.9 | 1,419 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 32.8 | 1,029 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 32.8 | 846 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3500 Ada Generation Laptop GPU | Q4_K - Medium | 32.8 | 1,337 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 32.8 | 1,028 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 32.7 | 1,002 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 32.4 | 1,016 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 32.4 | 489 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 32.2 | 378 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 31.9 | 3,161 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 31.7 | 2,663 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 31.6 | 485 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (64-core GPU) | Q4_K - Medium | 31.5 | 366 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 Ada Generation | Q4_K - Medium | 31.4 | 1,408 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 31.3 | 1,287 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 31.3 | 378 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 31.2 | 378 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 31.1 | 2,195 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 31.1 | 360 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 31.1 | 874 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 31.1 | 359 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 31.0 | 351 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 30.8 | 343 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 30.8 | 368 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 30.8 | 338 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 30.7 | 1,169 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 30.5 | 1,161 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 30.5 | 1,245 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 30.5 | 349 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 30.4 | 310 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 SUPER | Q4_K - Medium | 29.9 | 2,424 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 29.7 | 835 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 29.6 | 339 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 29.5 | 802 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 29.4 | 832 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 29.3 | 324 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 29.2 | 799 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 29.1 | 713 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 29.1 | 321 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5080 | Q4_K - Medium | 28.9 | 1,725 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla V100-SXM2-32GB | Q4_K - Medium | 28.9 | 907 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 Laptop GPU | Q4_K - Medium | 28.8 | 1,004 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 | Q4_K - Medium | 28.6 | 793 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 Ada Generation | Q4_K - Medium | 28.6 | 1,426 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 28.5 | 302 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 28.5 | 309 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 Laptop GPU | Q4_K - Medium | 28.4 | 821 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Laptop GPU | Q4_K - Medium | 28.4 | 779 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 28.3 | 308 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 28.1 | 304 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 6000 Ada | Q4_K - Medium | 28.1 | 2,936 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 28.1 | 832 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 28.0 | 328 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 27.9 | 502 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (48-core GPU) | Q4_K - Medium | 27.9 | 290 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (48-core GPU) | Q4_K - Medium | 27.7 | 289 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 27.6 | 2,261 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 27.6 | 752 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 27.4 | 2,671 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Laptop GPU | Q4_K - Medium | 27.4 | 793 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 27.4 | 783 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 | Q4_K - Medium | 27.4 | 761 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 27.3 | 280 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 27.2 | 2,045 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 27.1 | 798 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 27.1 | 1,692 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 27.1 | 1,205 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 26.9 | 2,262 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 26.9 | 791 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 26.9 | 1,266 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 26.9 | 300 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 26.9 | 300 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 26.8 | 301 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 26.8 | 1,260 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 26.7 | 309 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 26.7 | 1,737 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 Laptop GPU | Q4_K - Medium | 26.7 | 858 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 26.6 | 1,840 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 26.6 | 301 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 26.6 | 1,018 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 26.5 | 299 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 26.4 | 277 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 26.4 | 299 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 26.2 | 1,260 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 26.1 | 2,626 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 26.0 | 803 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | Q4_K - Medium | 25.9 | 3,061 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 25.5 | 303 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 25.5 | 302 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (38-core GPU) | Q4_K - Medium | 25.2 | 252 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 SUPER | Q4_K - Medium | 24.9 | 1,793 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L40S | Q4_K - Medium | 24.9 | 2,867 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | Q4_K - Medium | 24.9 | 2,735 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 SFF Ada Generation | Q4_K - Medium | 24.9 | 1,037 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 24.8 | 268 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L40 | Q4_K - Medium | 24.7 | 2,713 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 SFF Ada Generation | Q4_K - Medium | 24.5 | 1,066 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 24.5 | 1,241 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q4_K - Medium | 24.5 | 612 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 24.3 | 1,234 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5090 | Q4_K - Medium | 24.3 | 2,451 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (20-core GPU) | Q4_K - Medium | 24.3 | 221 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q4_K - Medium | 24.2 | 574 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (20-core GPU) | Q4_K - Medium | 24.2 | 217 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 4500 Blackwell | Q4_K - Medium | 24.2 | 1,824 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (20-core GPU) | Q4_K - Medium | 24.0 | 217 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 24.0 | 296 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 5080 | Q4_K - Medium | 23.9 | 2,275 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (38-core GPU) | Q4_K - Medium | 23.8 | 246 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5080 | Q4_K - Medium | 23.7 | 2,291 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (20-core GPU) | Q4_K - Medium | 23.7 | 216 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (20-core GPU) | Q4_K - Medium | 23.6 | 216 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 23.6 | 287 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| L4 | Q4_K - Medium | 23.6 | 1,072 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 23.6 | 257 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L4 | Q4_K - Medium | 23.5 | 1,119 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2060 | Q4_K - Medium | 23.4 | 600 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| H100 80 GB | Q4_K - Medium | 23.4 | 2,609 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 23.4 | 1,205 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M2 Max (38-core GPU) | Q4_K - Medium | 23.4 | 232 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 23.2 | 1,482 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | Q4_K - Medium | 23.2 | 2,256 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | Q4_K - Medium | 23.1 | 2,118 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 23.1 | 255 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7800 XT | Q4_K - Medium | 23.0 | 302 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7800 XT | Q4_K - Medium | 23.0 | 302 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7800 XT | Q4_K - Medium | 23.0 | 316 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7800 XT | Q4_K - Medium | 22.7 | 301 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 22.6 | 1,284 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 22.4 | 235 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 XT | Q4_K - Medium | 22.3 | 517 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 22.3 | 248 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2060 | Q4_K - Medium | 22.2 | 589 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6900 XT | Q4_K - Medium | 22.2 | 593 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (16-core GPU) | Q4_K - Medium | 21.6 | 169 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell | Q4_K - Medium | 21.5 | 3,215 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | Q4_K - Medium | 21.5 | 2,942 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 21.4 | 203 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7800 XT | Q4_K - Medium | 21.3 | 171 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 21.1 | 205 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 Ti | Q4_K - Medium | 21.1 | 449 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 21.1 | 205 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2000 Ada | Q4_K - Medium | 21.0 | 733 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 21.0 | 207 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 20.9 | 204 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 20.9 | 1,129 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 20.9 | 199 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 20.9 | 203 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (30-core GPU) | Q4_K - Medium | 20.9 | 242 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 | Q4_K - Medium | 20.8 | 1,264 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| A100 80 GB | Q4_K - Medium | 20.8 | 1,438 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 20.7 | 203 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 20.7 | 201 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (30-core GPU) | Q4_K - Medium | 20.7 | 236 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 20.6 | 204 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (38-core GPU) | Q4_K - Medium | 20.6 | 226 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 Ti | Q4_K - Medium | 20.6 | 427 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (30-core GPU) | Q4_K - Medium | 20.5 | 230 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 XT | Q4_K - Medium | 20.5 | 505 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla P100-PCIE-16GB | Q4_K - Medium | 20.5 | 204 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 20.5 | 191 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 NVL | Q4_K - Medium | 20.5 | 2,357 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 NVL | Q4_K - Medium | 20.5 | 2,351 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 20.4 | 205 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2000 Ada | Q4_K - Medium | 20.3 | 748 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 20.2 | 158 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 20.2 | 262 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 Ti | Q4_K - Medium | 20.2 | 424 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 12GB | Q4_K - Medium | 20.0 | 522 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 20.0 | 1,174 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 XT | Q4_K - Medium | 20.0 | 492 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 20.0 | 1,127 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 20.0 | 1,128 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| A10 | Q4_K - Medium | 19.9 | 1,133 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 19.7 | 1,156 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 12GB | Q4_K - Medium | 19.7 | 531 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 Ti | Q4_K - Medium | 19.6 | 412 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 19.6 | 197 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 12GB | Q4_K - Medium | 19.5 | 489 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 | Q4_K - Medium | 19.5 | 404 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 Ti | Q4_K - Medium | 19.4 | 381 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 12GB | Q4_K - Medium | 19.3 | 499 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tesla P100-PCIE-16GB | Q4_K - Medium | 19.1 | 181 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 19.1 | 1,151 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (30-core GPU) | Q4_K - Medium | 19.0 | 222 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 | Q4_K - Medium | 18.9 | 417 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 12GB | Q4_K - Medium | 18.9 | 490 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 18.8 | 1,152 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 18.8 | 1,149 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 18.8 | 963 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 18.8 | 1,145 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (38-core GPU) | Q4_K - Medium | 18.8 | 191 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 18.8 | 1,138 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 18.7 | 1,157 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 18.7 | 211 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 18.6 | 208 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.6 | 194 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 | Q4_K - Medium | 18.5 | 408 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 18.5 | 188 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.3 | 197 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.3 | 185 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 18.2 | 1,106 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.2 | 197 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.2 | 187 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 18.2 | 1,078 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.2 | 194 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.2 | 187 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.1 | 195 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.1 | 189 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.1 | 195 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 18.1 | 178 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.0 | 192 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.0 | 196 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.0 | 192 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.0 | 191 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.0 | 192 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 18.0 | 189 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 17.9 | 187 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 17.9 | 188 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 17.9 | 180 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 17.9 | 192 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 17.9 | 1,102 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 17.8 | 184 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 17.8 | 189 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 17.8 | 180 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 17.7 | 178 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 17.6 | 1,072 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 17.6 | 159 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 17.6 | 187 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 17.5 | 160 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 17.4 | 156 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3500 Ada Generation Laptop GPU | Q4_K - Medium | 17.4 | 1,259 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 17.2 | 170 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 17.1 | 180 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 16.9 | 156 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 16.9 | 2,471 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 16.8 | 162 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 16.8 | 161 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 16.7 | 154 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 16.7 | 157 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 16.7 | 153 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 16.6 | 153 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 16.6 | 953 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 16.6 | 1,014 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 16.6 | 153 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 16.5 | 150 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 16.5 | 156 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5000 Ada Generation Laptop GPU | Q4_K - Medium | 16.4 | 952 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 16.1 | 151 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 15.8 | 1,018 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla P40 | Q4_K - Medium | 15.7 | 339 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6700 XT | Q4_K - Medium | 15.7 | 316 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro P5000 | Q4_K - Medium | 14.9 | 303 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (30-core GPU) | Q4_K - Medium | 14.5 | 149 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 14.4 | 257 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon Graphics | Q4_K - Medium | 14.4 | 52.9 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell | Q4_K - Medium | 14.3 | 1,947 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 14.2 | 1,740 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M2 Pro (19-core GPU) | Q4_K - Medium | 14.1 | 137 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 13.9 | 1,383 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 13.9 | 202 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 13.8 | 130 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon Graphics | Q4_K - Medium | 13.7 | 48.8 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 13.7 | 200 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L40S | Q4_K - Medium | 13.6 | 2,868 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 | Q4_K - Medium | 13.5 | 1,747 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4080 | Q4_K - Medium | 13.4 | 1,760 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 13.4 | 119 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 | Q4_K - Medium | 13.4 | 357 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 13.3 | 117 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 13.3 | 119 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla T4 | Q4_K - Medium | 13.3 | 450 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 13.3 | 118 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 13.1 | 1,973 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Instinct MI50/MI60 | Q4_K - Medium | 12.9 | 48.4 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 12.6 | 1,638 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 12.3 | 147 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 12.2 | 1,420 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 12.2 | 121 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 12.2 | 120 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 12.2 | 151 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 12.1 | 149 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 12.1 | 149 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 12.0 | 144 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 12.0 | 119 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 11.9 | 111 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 11.9 | 117 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 11.9 | 107 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 11.9 | 117 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 11.9 | 119 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 11.9 | 106 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 11.9 | 111 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 11.9 | 108 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 11.8 | 115 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 11.7 | 112 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 11.7 | 147 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 11.7 | 146 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 11.6 | 144 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 11.6 | 105 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla P40 | Q4_K - Medium | 11.5 | 245 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 11.2 | 141 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 11.2 | 99.2 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 11.1 | 109 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 11.0 | 94.8 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 10.9 | 116 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 SUPER | Q4_K - Medium | 10.9 | 1,299 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 10.8 | 93.9 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 10.8 | 93.2 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 10.8 | 92.5 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 10.8 | 88.0 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 10.4 | 88.9 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 9.9 | 102 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 9.8 | 94.2 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 9.6 | 96.3 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 9.6 | 95.8 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 9.6 | 94.3 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 9.5 | 547 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 9.5 | 91.6 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 9.4 | 93.7 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 9.2 | 93.3 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 9.2 | 90.4 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 8.5 | 73.6 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 8.1 | 71.2 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 8.1 | 74.8 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 8.0 | 70.5 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4500 Laptop GPU | Q4_K - Medium | 7.9 | 302 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 (8-core GPU) | Q4_K - Medium | 7.8 | 65.9 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 7.5 | 430 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 7.3 | 70.3 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (8-core GPU) | Q4_K - Medium | 7.3 | 60.5 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M2 (8-core GPU) | Q4_K - Medium | 7.0 | 60.0 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (8-core GPU) | Q4_K - Medium | 6.6 | 56.2 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 6.5 | 61.3 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 6.3 | 67.3 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 6.2 | 420 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 6.1 | 65.2 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 6.0 | 64.9 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 5.4 | 53.3 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 | Q4_K - Medium | 5.3 | 324 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3070 | Q4_K - Medium | 5.1 | 376 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 4.8 | 40.7 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 4.8 | 40.6 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 SUPER | Q4_K - Medium | 4.5 | 294 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 4.3 | 45.8 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Laptop GPU | Q4_K - Medium | 3.8 | 289 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 3.5 | 327 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L40S-8Q | Q4_K - Medium | 3.5 | 343 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6950 XT | Q4_K - Medium | 3.3 | 550 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6950 XT | Q4_K - Medium | 3.3 | 549 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 3.0 | 99.1 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6900 XT | Q4_K - Medium | 3.0 | 496 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 | Q4_K - Medium | 2.5 | 368 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 2000 Blackwell Generation Laptop GPU | Q4_K - Medium | 2.1 | 230 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 1.9 | 57.3 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| GTX 1050 Ti | Q4_K - Medium | 1.2 | 36.9 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 1.1 | 82.9 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
Ollama
No measured Ollama results yet. Submit yours →
MLX
No measured MLX results yet. Submit yours →
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.