Models /
Meta Llama 3.1 8B Instruct
447results
126devices
8.03Bparams
138fastest gen tok/s
llama.cpp
| Hardware | Quant | Gen tok/s | Prompt tok/s | Source | Context tested | Build | |
|---|---|---|---|---|---|---|---|
| RTX PRO 6000 Blackwell | Q4_K - Medium | 138 | 8,479 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 125 | 7,871 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 122 | 6,657 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 6000 Ada | Q4_K - Medium | 121 | 6,808 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 PCIe | Q4_K - Medium | 120 | 5,202 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 120 | 7,594 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 117 | 3,895 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 117 | 3,926 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 115 | 6,909 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 113 | 3,991 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 Ti | Q4_K - Medium | 111 | 4,120 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| A100 80 GB | Q4_K - Medium | 110 | 3,571 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 110 | 3,895 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 110 | 3,898 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 109 | 3,858 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 Ti | Q4_K - Medium | 109 | 3,928 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 106 | 8,441 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 104 | 3,739 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 103 | 3,552 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| RTX 3090 | Q4_K - Medium | 103 | 3,637 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 103 | 3,511 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 103 | 7,398 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 101 | 3,454 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 101 | 3,307 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 98.5 | 7,159 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 94.5 | 4,546 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 93.3 | 2,950 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 92.5 | 3,140 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 92.4 | 2,796 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 92.1 | 3,156 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 | Q4_K - Medium | 92.1 | 4,698 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 91.8 | 3,860 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A6000 | Q4_K - Medium | 90.5 | 3,447 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 | Q4_K - Medium | 90.1 | 4,789 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 | Q4_K - Medium | 89.9 | 2,890 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 89.7 | 2,880 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A5000 | Q4_K - Medium | 88.3 | 2,891 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla V100-SXM2-32GB | Q4_K - Medium | 88.0 | 2,204 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 85.4 | 3,876 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 84.2 | 7,015 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 84.0 | 3,407 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3070 Ti | Q4_K - Medium | 83.2 | 2,509 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 83.1 | 3,641 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 | Q4_K - Medium | 82.8 | 4,888 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 82.5 | 4,285 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 81.3 | 3,482 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| A40 | Q4_K - Medium | 80.6 | 3,270 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 80.5 | 6,998 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3080 | Q4_K - Medium | 80.3 | 3,123 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4080 SUPER | Q4_K - Medium | 80.1 | 5,033 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 79.8 | 4,323 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 79.7 | 3,363 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3080 Ti | Q4_K - Medium | 79.2 | 3,204 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3080 | Q4_K - Medium | 79.0 | 3,125 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX A4500 | Q4_K - Medium | 78.2 | 2,513 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4500 | Q4_K - Medium | 77.6 | 2,484 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 SUPER | Q4_K - Medium | 77.3 | 4,924 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 77.2 | 4,006 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 76.6 | 3,028 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 76.4 | 3,128 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 4070 | Q4_K - Medium | 76.3 | 3,192 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 75.7 | 2,792 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 75.5 | 2,891 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3080 | Q4_K - Medium | 74.8 | 3,079 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5070 | Q4_K - Medium | 74.8 | 3,495 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Quadro RTX 8000 | Q4_K - Medium | 74.0 | 1,677 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 Ti | Q4_K - Medium | 74.0 | 2,102 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Titan V | Q4_K - Medium | 73.3 | 1,823 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 Ti | Q4_K - Medium | 71.8 | 1,952 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 Laptop GPU | Q4_K - Medium | 71.3 | 3,104 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 5070 Ti Laptop GPU | Q4_K - Medium | 70.0 | 2,937 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 5090 | Q4_K - Medium | 70.0 | 5,758 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 69.1 | 3,985 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 68.2 | 2,836 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 68.0 | 3,916 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 SUPER | Q4_K - Medium | 68.0 | 3,682 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 67.3 | 1,974 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti Laptop GPU | Q4_K - Medium | 66.3 | 3,075 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 66.3 | 4,258 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3070 | Q4_K - Medium | 66.1 | 2,204 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 65.9 | 2,150 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 65.4 | 2,251 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 65.3 | 1,870 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 65.1 | 1,898 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro RTX 6000 | Q4_K - Medium | 64.9 | 1,891 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 64.7 | 1,468 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti Laptop GPU | Q4_K - Medium | 64.2 | 2,781 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 63.8 | 2,084 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3070 | Q4_K - Medium | 63.4 | 2,184 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 63.4 | 1,830 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 63.3 | 1,062 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 62.7 | 1,119 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 62.7 | 2,198 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 62.6 | 2,387 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Ultra (80-core GPU) | Q4_K - Medium | 62.6 | 1,100 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Max (32-core GPU) | Q4_K - Medium | 62.2 | 635 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 61.9 | 2,495 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 61.7 | 1,729 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Max (32-core GPU) | Q4_K - Medium | 61.6 | 630 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 61.5 | 2,535 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 60.6 | 2,011 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 60.6 | 2,427 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 60.5 | 2,160 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 60.3 | 2,145 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 60.0 | 1,989 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2080 SUPER | Q4_K - Medium | 59.8 | 1,693 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3070 | Q4_K - Medium | 59.7 | 1,745 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M2 Ultra (60-core GPU) | Q4_K - Medium | 59.5 | 703 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 59.2 | 1,905 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 2080 | Q4_K - Medium | 59.1 | 1,540 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 59.1 | 1,949 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 59.0 | 1,963 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 58.9 | 1,945 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 58.8 | 698 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 58.8 | 1,932 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 58.6 | 1,843 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 58.6 | 1,962 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 58.5 | 1,910 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4090 | Q4_K - Medium | 58.3 | 5,925 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2070 SUPER | Q4_K - Medium | 57.8 | 1,460 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 57.5 | 1,545 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3500 Ada Generation Laptop GPU | Q4_K - Medium | 57.4 | 2,457 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2070 SUPER | Q4_K - Medium | 57.4 | 1,428 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 2080 | Q4_K - Medium | 57.1 | 1,593 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4000 Ada Generation | Q4_K - Medium | 56.5 | 2,617 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 56.1 | 704 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 55.7 | 3,834 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 55.6 | 672 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2070 | Q4_K - Medium | 55.5 | 1,316 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 55.5 | 655 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 55.3 | 2,235 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 54.8 | 652 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4080 SUPER | Q4_K - Medium | 54.4 | 4,461 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (64-core GPU) | Q4_K - Medium | 54.4 | 675 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 54.4 | 1,861 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 54.2 | 663 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (64-core GPU) | Q4_K - Medium | 54.2 | 660 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Laptop GPU | Q4_K - Medium | 53.9 | 2,052 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 2070 | Q4_K - Medium | 53.3 | 1,248 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 53.1 | 627 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 53.1 | 2,089 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 53.1 | 1,487 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 53.0 | 1,506 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 52.8 | 1,503 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 52.4 | 636 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 52.4 | 1,478 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 Laptop GPU | Q4_K - Medium | 52.3 | 1,885 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 | Q4_K - Medium | 52.2 | 1,429 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3080 | Q4_K - Medium | 52.1 | 2,963 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3060 | Q4_K - Medium | 52.0 | 1,482 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2070 | Q4_K - Medium | 51.7 | 1,216 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 51.6 | 607 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| RTX 2070 | Q4_K - Medium | 51.5 | 1,216 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 51.4 | 853 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 2060 SUPER | Q4_K - Medium | 51.3 | 1,223 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 6000 Ada | Q4_K - Medium | 51.3 | 5,488 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 51.2 | 590 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 51.2 | 888 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 Ada Generation | Q4_K - Medium | 50.6 | 2,599 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6900 XT | Q4_K - Medium | 50.3 | 1,170 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2070 | Q4_K - Medium | 49.9 | 1,263 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 49.9 | 3,833 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 49.7 | 1,400 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 49.6 | 583 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 49.6 | 1,555 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 49.6 | 2,130 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 | Q4_K - Medium | 49.5 | 1,515 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 49.3 | 2,247 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 49.1 | 3,366 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 49.1 | 2,267 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (48-core GPU) | Q4_K - Medium | 49.1 | 535 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 49.1 | 3,661 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (48-core GPU) | Q4_K - Medium | 48.8 | 535 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 48.8 | 3,305 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 48.8 | 3,342 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Ultra (48-core GPU) | Q4_K - Medium | 48.7 | 534 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 48.7 | 584 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 48.7 | 584 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 48.6 | 3,827 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 48.6 | 4,023 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 48.6 | 577 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 48.3 | 580 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 48.3 | 3,239 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 48.2 | 3,231 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 48.1 | 3,061 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 48.1 | 3,235 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 48.0 | 3,240 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 47.9 | 3,227 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 47.7 | 562 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 47.5 | 553 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 47.2 | 551 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | Q4_K - Medium | 46.9 | 5,576 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | Q4_K - Medium | 46.9 | 5,573 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (38-core GPU) | Q4_K - Medium | 46.4 | 484 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 46.0 | 583 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 46.0 | 1,885 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| L40S | Q4_K - Medium | 45.9 | 5,328 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 | Q4_K - Medium | 45.8 | 1,748 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 45.8 | 4,682 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 45.6 | 591 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 45.5 | 1,439 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX A5000 Laptop GPU | Q4_K - Medium | 45.5 | 1,497 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 SUPER | Q4_K - Medium | 45.5 | 3,371 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A4000 | Q4_K - Medium | 45.4 | 1,881 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 45.4 | 2,231 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 45.4 | 2,224 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 7900 XTX | Q4_K - Medium | 45.3 | 885 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 45.2 | 1,450 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| L40 | Q4_K - Medium | 45.1 | 5,065 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 45.1 | 1,410 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5080 | Q4_K - Medium | 44.9 | 4,428 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (20-core GPU) | Q4_K - Medium | 44.9 | 411 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 44.9 | 2,182 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX PRO 6000 Blackwell Max-Q Workstation Edition | Q4_K - Medium | 44.9 | 5,483 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 44.8 | 2,217 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (38-core GPU) | Q4_K - Medium | 44.7 | 474 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 SFF Ada Generation | Q4_K - Medium | 44.6 | 2,013 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 44.5 | 1,426 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M5 Pro (20-core GPU) | Q4_K - Medium | 44.5 | 410 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q6_K | 44.5 | 1,201 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4000 SFF Ada Generation | Q4_K - Medium | 44.1 | 1,995 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (16-core GPU) | Q4_K - Medium | 43.7 | 341 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 43.5 | 1,265 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Laptop GPU | Q4_K - Medium | 43.4 | 1,695 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5090 | Q4_K - Medium | 43.2 | 6,307 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 Pro (16-core GPU) | Q4_K - Medium | 43.2 | 336 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 43.0 | 526 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| L4 | Q4_K - Medium | 42.9 | 2,056 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 42.8 | 1,843 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 Ti | Q4_K - Medium | 42.6 | 1,279 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| H100 80 GB | Q4_K - Medium | 42.4 | 4,687 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 42.2 | 475 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 41.9 | 2,617 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3070 Laptop GPU | Q4_K - Medium | 41.9 | 1,176 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M4 Max (40-core GPU) | Q4_K - Medium | 41.8 | 478 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 | Q4_K - Medium | 41.2 | 1,708 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro RTX 4000 | Q4_K - Medium | 41.2 | 1,096 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 5060 Ti | Q4_K - Medium | 41.1 | 2,163 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Laptop GPU | Q4_K - Medium | 40.9 | 2,108 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4070 Laptop GPU | Q4_K - Medium | 40.9 | 2,153 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4060 | Q4_K - Medium | 40.7 | 1,702 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 2000 Blackwell Generation Laptop GPU | Q4_K - Medium | 40.4 | 1,453 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4060 Laptop GPU | Q4_K - Medium | 40.3 | 1,591 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6900 XT | Q4_K - Medium | 40.3 | 1,028 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 39.8 | 1,250 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 4060 | Q4_K - Medium | 39.6 | 1,624 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6800 XT | Q4_K - Medium | 39.2 | 975 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 7800 XT | Q4_K - Medium | 38.9 | 524 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro RTX 4000 | Q4_K - Medium | 38.8 | 889 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 38.4 | 392 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 38.4 | 1,094 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q8_0 | 38.3 | 1,600 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 XT | Q4_K - Medium | 38.2 | 951 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 38.1 | 384 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2060 | Q4_K - Medium | 38.1 | 960 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 37.9 | 385 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2000 Ada | Q4_K - Medium | 37.9 | 1,328 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2000 Ada | Q4_K - Medium | 37.9 | 1,373 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M3 Max (30-core GPU) | Q4_K - Medium | 37.7 | 457 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 37.7 | 383 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 37.7 | 388 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (30-core GPU) | Q4_K - Medium | 37.7 | 442 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 37.6 | 389 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 37.5 | 369 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (30-core GPU) | Q4_K - Medium | 37.3 | 445 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| A100 80 GB | Q4_K - Medium | 37.1 | 2,601 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 37.1 | 2,110 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| H100 NVL | Q4_K - Medium | 37.1 | 4,299 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 | Q4_K - Medium | 37.0 | 2,360 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 2000 Ada | Q4_K - Medium | 36.8 | 1,370 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q4_K - Medium | 36.7 | 1,050 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 36.6 | 2,060 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 36.5 | 2,102 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| A10 | Q4_K - Medium | 36.5 | 2,085 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 36.5 | 2,106 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 3070 | Q4_K - Medium | 36.4 | 1,821 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 12GB | Q4_K - Medium | 35.7 | 976 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 XT | Q4_K - Medium | 35.5 | 926 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 35.4 | 1,961 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3050 | Q4_K - Medium | 35.0 | 965 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| DGX Spark | Q4_K - Medium | 34.9 | 2,027 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q5_K - Medium | 34.9 | 1,157 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 34.8 | 2,042 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 34.8 | 1,050 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3050 | Q4_K - Medium | 34.5 | 979 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 34.2 | 367 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 7800 XT | Q4_K - Medium | 34.1 | 276 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M4 Max (32-core GPU) | Q4_K - Medium | 33.7 | 432 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 33.7 | 1,951 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 33.6 | 350 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla P100-PCIE-16GB | Q4_K - Medium | 33.5 | 427 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 33.3 | 1,955 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DGX Spark | Q4_K - Medium | 33.2 | 2,035 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 33.1 | 366 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | Q6_K | 33.1 | 1,016 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 33.1 | 2,044 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 33.1 | 2,475 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 33.0 | 367 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.9 | 362 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.8 | 369 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.8 | 366 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.7 | 351 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.7 | 363 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.7 | 352 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.7 | 360 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.6 | 364 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.6 | 362 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.5 | 357 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.5 | 358 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 32.3 | 338 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 Max (24-core GPU) | Q4_K - Medium | 32.1 | 306 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 32.1 | 358 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 31.8 | 329 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4050 Laptop GPU | Q4_K - Medium | 31.7 | 1,217 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M4 Pro (20-core GPU) | Q4_K - Medium | 31.6 | 351 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Max (32-core GPU) | Q4_K - Medium | 31.6 | 312 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Max (30-core GPU) | Q4_K - Medium | 31.2 | 325 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Tesla P100-PCIE-16GB | Q4_K - Medium | 31.1 | 372 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 4050 Laptop GPU | Q4_K - Medium | 31.0 | 1,185 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 30.9 | 302 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6800 | Q4_K - Medium | 30.9 | 737 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 30.8 | 1,831 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 30.7 | 2,112 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 30.7 | 291 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 30.6 | 302 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 30.5 | 303 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 30.5 | 302 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 30.4 | 292 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 30.2 | 302 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3060 Laptop GPU | Q4_K - Medium | 30.0 | 891 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti SUPER | Q4_K - Medium | 29.6 | 3,313 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M4 Pro (16-core GPU) | Q4_K - Medium | 29.5 | 298 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| RTX 2000 Ada Generation Laptop GPU | Q4_K - Medium | 29.5 | 1,151 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6900 XT | Q8_0 | 29.5 | 1,389 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 | Q4_K - Medium | 28.7 | 758 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX A2000 8GB Laptop GPU | Q4_K - Medium | 27.6 | 791 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX A2000 8GB Laptop GPU | Q4_K - Medium | 27.5 | 789 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6700 XT | Q4_K - Medium | 27.3 | 607 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3050 6GB Laptop GPU | Q4_K - Medium | 26.9 | 899 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M2 Pro (19-core GPU) | Q4_K - Medium | 26.3 | 262 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (19-core GPU) | Q4_K - Medium | 26.3 | 262 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3500 Ada Generation Laptop GPU | Q4_K - Medium | 26.1 | 2,433 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3050 | Q4_K - Medium | 26.0 | 760 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6650 XT | Q4_K - Medium | 25.9 | 563 | LocalScore ↗ | 512+128 | llamafile 0.9.1 | Details → |
| RTX 3050 | Q4_K - Medium | 25.8 | 797 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 4090 | Q4_K - Medium | 25.8 | 4,014 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 25.8 | 383 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 Ti | Q4_K - Medium | 25.7 | 2,862 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 5700 XT | Q4_K - Medium | 25.4 | 256 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Max (40-core GPU) | Q4_K - Medium | 25.1 | 371 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3090 | Q4_K - Medium | 25.1 | 2,540 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 25.0 | 229 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 24.5 | 224 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 24.3 | 226 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Laptop GPU | Q4_K - Medium | 24.3 | 809 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| DGX Spark | Q4_K - Medium | 24.1 | 1,510 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 5070 Ti | Q4_K - Medium | 24.0 | 3,083 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 23.8 | 209 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 Pro (16-core GPU) | Q4_K - Medium | 23.6 | 219 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 | Q4_K - Medium | 23.4 | 857 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 22.7 | 207 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4070 SUPER | Q4_K - Medium | 22.6 | 2,594 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX PRO 1000 Blackwell Generation Laptop GPU | Q4_K - Medium | 22.5 | 1,012 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 1000 Blackwell Generation Laptop GPU | Q4_K - Medium | 22.5 | 984 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 22.4 | 211 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 22.3 | 211 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 22.2 | 230 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 22.2 | 210 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 22.1 | 284 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 22.0 | 224 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 22.0 | 205 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 22.0 | 205 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 21.9 | 202 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 21.6 | 187 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6600 XT | Q4_K - Medium | 21.5 | 604 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 3050 6GB Laptop GPU | Q4_K - Medium | 21.5 | 681 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 21.5 | 223 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 21.2 | 195 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 1000 Blackwell Generation Laptop GPU | Q4_K - Medium | 21.0 | 944 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (16-core GPU) | Q4_K - Medium | 21.0 | 202 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6600 | Q4_K - Medium | 20.8 | 418 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M3 Pro (18-core GPU) | Q4_K - Medium | 20.8 | 279 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX 4060 Ti | Q4_K - Medium | 20.6 | 1,929 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 20.4 | 181 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 | Q4_K - Medium | 20.2 | 651 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1080 | Q4_K - Medium | 20.2 | 653 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 20.2 | 180 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 20.0 | 173 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 Pro (14-core GPU) | Q4_K - Medium | 19.8 | 170 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M5 (10-core GPU) | Q4_K - Medium | 19.6 | 177 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M3 Pro (14-core GPU) | Q4_K - Medium | 19.1 | 200 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (8-core GPU) | Q4_K - Medium | 18.8 | 148 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 18.1 | 185 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (8-core GPU) | Q4_K - Medium | 17.9 | 149 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 17.8 | 182 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 17.7 | 186 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 17.7 | 185 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6900 XT | F16 | 17.7 | 1,243 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 17.6 | 188 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 17.4 | 180 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 17.1 | 178 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 16.9 | 173 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 16.6 | 152 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 16.6 | 159 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 16.4 | 172 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 15.9 | 146 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple A18 Pro 2P+4E+5GPU | Q4_K - Medium | 15.8 | 116 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 15.6 | 157 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (8-core GPU) | Q4_K - Medium | 15.3 | 134 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 15.2 | 144 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 14.9 | 143 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 14.7 | 136 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 14.7 | 142 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 14.6 | 134 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 14.6 | 134 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M2 (10-core GPU) | Q4_K - Medium | 14.0 | 135 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (8-core GPU) | Q4_K - Medium | 13.7 | 125 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 13.7 | 137 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 13.6 | 137 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| RTX PRO 1000 Blackwell Generation Laptop GPU | Q4_K - Medium | 13.4 | 685 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 13.4 | 109 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 13.0 | 120 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M2 (8-core GPU) | Q4_K - Medium | 12.9 | 114 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M4 (10-core GPU) | Q4_K - Medium | 11.7 | 121 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 11.1 | 123 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 10.8 | 121 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1660 SUPER | Q4_K - Medium | 10.0 | 133 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 9.7 | 101 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 9.7 | 88.0 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M3 (10-core GPU) | Q4_K - Medium | 9.7 | 98.5 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Apple M1 (7-core GPU) | Q4_K - Medium | 9.1 | 79.9 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| GTX 1660 | Q4_K - Medium | 9.0 | 129 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Apple M1 (8-core GPU) | Q4_K - Medium | 8.9 | 84.0 | LocalScore ↗ | 4,352 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6950 XT | Q4_K - Medium | 6.2 | 1,031 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6950 XT | Q4_K - Medium | 6.1 | 1,017 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 5.6 | 186 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 7600 | Q4_K - Medium | 5.5 | 317 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6700 XT | Q4_K - Medium | 5.4 | 587 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon RX 6600 | Q4_K - Medium | 5.2 | 424 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| RTX 3050 Laptop GPU | Q4_K - Medium | 4.8 | 280 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon RX 6600 | Q4_K - Medium | 4.7 | 415 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| RTX 3050 Ti Laptop GPU | Q4_K - Medium | 4.3 | 290 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 3.8 | 112 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 3.5 | 110 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Radeon 780M Graphics | Q4_K - Medium | 3.3 | 93.9 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| GTX 1650 Ti | Q4_K - Medium | 2.8 | 65.6 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| Quadro P620 | Q4_K - Medium | 1.3 | 45.4 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
Ollama
No measured Ollama results yet. Submit yours →
MLX
No measured MLX results yet. Submit yours →
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.