Hardware
- RTX 4050 Laptop GPU234 results · up to 155 tok/s
- RTX 3090113 results · up to 377 tok/s
- Apple M4 Pro (20-core GPU)73 results · up to 123 tok/s
- Apple M4 Max (40-core GPU)70 results · up to 191 tok/s
- Apple M4 (10-core GPU)54 results · up to 78.6 tok/s
- RTX 5070 Ti47 results · up to 260 tok/s
- Apple M1 Max (32-core GPU)46 results · up to 128 tok/s
- RTX 306045 results · up to 214 tok/s
- DGX Spark41 results · up to 158 tok/s
- RTX 308040 results · up to 330 tok/s
- RTX A400038 results · up to 241 tok/s
- RTX 409038 results · up to 432 tok/s
- RTX 4060 Ti38 results · up to 219 tok/s
- RTX 4070 Ti SUPER37 results · up to 342 tok/s
- RTX 509036 results · up to 360 tok/s
- RTX 5060 Ti35 results · up to 309 tok/s
- Apple M4 Pro (16-core GPU)35 results · up to 112 tok/s
- Apple M4 Max (32-core GPU)34 results · up to 171 tok/s
- RTX 307031 results · up to 263 tok/s
- AMD Radeon RX 6900 XT30 results · up to 113 tok/s
- Apple M1 Pro (16-core GPU)28 results · up to 82.5 tok/s
- RTX 3060 Laptop GPU28 results · up to 210 tok/s
- Apple M3 (10-core GPU)26 results · up to 1,564 tok/s
- RTX 2080 Ti23 results · up to 260 tok/s
- RTX 4070 Ti22 results · up to 309 tok/s
- AMD Radeon RX 7800 XT22 results · up to 125 tok/s
- RTX 3060 Ti22 results · up to 244 tok/s
- Apple M2 (10-core GPU)21 results · up to 60.0 tok/s
- Apple M3 Max (40-core GPU)20 results · up to 153 tok/s
- Apple M2 Pro (16-core GPU)20 results · up to 91.5 tok/s
- Apple M1 Pro (14-core GPU)20 results · up to 72.9 tok/s
- GTX 1080 Ti19 results · up to 153 tok/s
- Apple M3 Pro (18-core GPU)18 results · up to 92.6 tok/s
- Apple M3 Pro (14-core GPU)18 results · up to 88.7 tok/s
- Apple M1 Max (24-core GPU)18 results · up to 106 tok/s
- Apple M1 (8-core GPU)18 results · up to 47.5 tok/s
- GTX 1660 SUPER16 results · up to 63.9 tok/s
- GTX 1060 6GB16 results · up to 86.5 tok/s
- Apple M1 (7-core GPU)15 results · up to 38.9 tok/s
- AMD Radeon RX 7900 XTX15 results · up to 195 tok/s
- Apple M3 Ultra (80-core GPU)15 results · up to 179 tok/s
- AMD Radeon 780M Graphics15 results · up to 12.1 tok/s
- RTX A500015 results · up to 313 tok/s
- Apple M2 Ultra (76-core GPU)14 results · up to 109 tok/s
- RTX 3080 Ti14 results · up to 332 tok/s
- Apple M5 (10-core GPU)14 results · up to 101 tok/s
- RTX 305013 results · up to 108 tok/s
- RTX 508013 results · up to 192 tok/s
- Apple M2 (8-core GPU)13 results · up to 52.9 tok/s
- RTX 4080 SUPER13 results · up to 247 tok/s
- RTX 4070 SUPER12 results · up to 238 tok/s
- Apple M2 Max (38-core GPU)12 results · up to 153 tok/s
- AMD Radeon RX 6800 XT12 results · up to 118 tok/s
- RTX 408012 results · up to 352 tok/s
- Apple M2 Pro (19-core GPU)12 results · up to 104 tok/s
- AMD Radeon RX 6650 XT12 results · up to 109 tok/s
- Apple M3 Max (30-core GPU)12 results · up to 133 tok/s
- P106-10011 results · up to 78.8 tok/s
- RTX PRO 6000 Blackwell11 results · up to 374 tok/s
- RTX 2000 Ada11 results · up to 173 tok/s
- Apple M1 Ultra (48-core GPU)11 results · up to 139 tok/s
- Apple M5 Pro (20-core GPU)10 results · up to 176 tok/s
- Quadro RTX 600010 results · up to 253 tok/s
- Apple M4 (8-core GPU)10 results · up to 66.3 tok/s
- AMD Radeon RX 6700 XT10 results · up to 103 tok/s
- Tesla P4010 results · up to 118 tok/s
- RTX PRO 6000 Blackwell Max-Q Workstation Edition10 results · up to 109 tok/s
- RTX 206010 results · up to 168 tok/s
- AMD Radeon RX 680010 results · up to 101 tok/s
- RTX 3090 Ti10 results · up to 372 tok/s
- Apple M2 Ultra (60-core GPU)9 results · up to 176 tok/s
- RTX A2000 12GB9 results · up to 155 tok/s
- Apple M2 Max (30-core GPU)9 results · up to 135 tok/s
- RTX 40709 results · up to 283 tok/s
- RTX 6000 Ada9 results · up to 416 tok/s
- Apple M1 Ultra (64-core GPU)9 results · up to 154 tok/s
- RTX 5070 Ti Laptop GPU8 results · up to 186 tok/s
- RTX 2070 SUPER8 results · up to 212 tok/s
- AMD Radeon RX 6600 XT8 results · up to 53.9 tok/s
- RTX A45008 results · up to 288 tok/s
- RTX 4060 Laptop GPU8 results · up to 192 tok/s
- A100 80 GB8 results · up to 308 tok/s
- RTX 20707 results · up to 196 tok/s
- RTX 50707 results · up to 128 tok/s
- RTX 40607 results · up to 190 tok/s
- AMD Radeon RX 66007 results · up to 72.9 tok/s
- GTX 1050 Ti7 results · up to 46.7 tok/s
- Quadro RTX 80007 results · up to 250 tok/s
- RTX 4000 SFF Ada Generation7 results · up to 191 tok/s
- L40S7 results · up to 361 tok/s
- L47 results · up to 190 tok/s
- A407 results · up to 285 tok/s
- Titan V7 results · up to 232 tok/s
- GTX 1070 Ti7 results · up to 117 tok/s
- RTX 4070 Laptop GPU6 results · up to 196 tok/s
- Tesla V100-SXM2-32GB6 results · up to 272 tok/s
- Tesla P100-PCIE-16GB6 results · up to 124 tok/s
- Apple M5 Max (32-core GPU)6 results · up to 230 tok/s
- AMD Radeon Graphics6 results · up to 72.6 tok/s
- H100 NVL6 results · up to 88.7 tok/s
- RTX 3080 Laptop GPU6 results · up to 157 tok/s
- RTX 4000 Ada Generation6 results · up to 237 tok/s
- H100 80 GB6 results · up to 281 tok/s
- AMD Radeon RX 9070 XT6 results · up to 165 tok/s
- GTX 10805 results · up to 75.4 tok/s
- RTX 20805 results · up to 222 tok/s
- GTX 10705 results · up to 103 tok/s
- Radeon RX 7900 XT5 results · up to 132 tok/s
- AMD Radeon RX 6950 XT5 results · up to 6.2 tok/s
- Quadro P20005 results · up to 58.3 tok/s
- RTX 3500 Ada Generation Laptop GPU5 results · up to 181 tok/s
- RTX A20005 results · up to 117 tok/s
- RTX A5000 Laptop GPU5 results · up to 202 tok/s
- L405 results · up to 154 tok/s
- RTX A60005 results · up to 315 tok/s
- Quadro P6204 results · up to 26.6 tok/s
- RTX 3070 Laptop GPU4 results · up to 124 tok/s
- RTX 3050 6GB Laptop GPU4 results · up to 108 tok/s
- Orin4 results · up to 47.8 tok/s
- RTX PRO 1000 Blackwell Generation Laptop GPU4 results · up to 22.5 tok/s
- H2004 results · up to 99.0 tok/s
- Quadro RTX 40004 results · up to 67.6 tok/s
- GTX 10504 results · up to 41.5 tok/s
- AMD Radeon PRO V6204 results · up to 102 tok/s
- RTX 4090 D4 results · up to 359 tok/s
- RTX 3050 Ti Laptop GPU4 results · up to 142 tok/s
- AMD Radeon RX 5700 XT4 results · up to 74.4 tok/s
- Apple M3 Ultra (60-core GPU)4 results · up to 88.4 tok/s
- RTX 2060 SUPER4 results · up to 195 tok/s
- Tesla V1004 results · up to 142 tok/s
- A304 results · up to 132 tok/s
- Tesla P1004 results · up to 66.5 tok/s
- Tesla M404 results · up to 41.7 tok/s
- Tesla P44 results · up to 33.3 tok/s
- Tesla K804 results · up to 14.3 tok/s
- Tesla V100-SXM2-16GB3 results · up to 278 tok/s
- RTX 5090 Laptop GPU3 results · up to 182 tok/s
- Apple M5 Pro (16-core GPU)3 results · up to 43.7 tok/s
- Apple M5 Max (40-core GPU)3 results · up to 37.5 tok/s
- GTX 16503 results · up to 23.8 tok/s
- GTX 1650 Ti3 results · up to 20.7 tok/s
- RTX 2080 SUPER3 results · up to 186 tok/s
- L40S-8Q3 results · up to 62.8 tok/s
- GTX 1060 3GB3 results · up to 98.7 tok/s
- RTX PRO 2000 Blackwell Generation Laptop GPU3 results · up to 131 tok/s
- AMD Radeon RX 7600 XT3 results · up to 57.1 tok/s
- Tesla T43 results · up to 46.4 tok/s
- RTX A2000 8GB Laptop GPU3 results · up to 109 tok/s
- GTX 16603 results · up to 41.4 tok/s
- A103 results · up to 93.5 tok/s
- Titan Xp3 results · up to 160 tok/s
- H100 PCIe3 results · up to 335 tok/s
- RTX PRO 4500 Blackwell2 results · up to 169 tok/s
- P106-0902 results · up to 42.1 tok/s
- Apple A18 Pro 2P+4E+5GPU2 results · up to 34.0 tok/s
- Apple M3 (8-core GPU)2 results · up to 60.3 tok/s
- Quadro P40002 results · up to 94.3 tok/s
- TITAN RTX2 results · up to 43.3 tok/s
- RTX 5000 Ada Generation Laptop GPU2 results · up to 21.3 tok/s
- RTX A4500 Laptop GPU2 results · up to 72.5 tok/s
- RTX A2000 Laptop GPU2 results · up to 106 tok/s
- GTX 1660 Ti2 results · up to 20.1 tok/s
- RTX 3070 Ti2 results · up to 297 tok/s
- RTX PRO 4000 Blackwell2 results · up to 136 tok/s
- RTX A4000 Ada2 results · up to 61.8 tok/s
- GTX 10602 results · up to 28.2 tok/s
- Quadro T10002 results · up to 27.8 tok/s
- Quadro P10002 results · up to 14.0 tok/s
- A1002 results · up to 171 tok/s
- AMD Radeon AI Pro R97002 results · up to 139 tok/s
- AMD Radeon RX 90702 results · up to 120 tok/s
- AMD Radeon RX 7900 GRE2 results · up to 116 tok/s
- Intel Arc Pro B702 results · up to 114 tok/s
- AMD Radeon Instinct MI502 results · up to 118 tok/s
- AMD Radeon Pro VII2 results · up to 108 tok/s
- AMD Radeon VII2 results · up to 111 tok/s
- AMD Radeon Pro W6800X Duo2 results · up to 100 tok/s
- AMD Radeon RX 6750 XT2 results · up to 82.3 tok/s
- Apple M4 Max2 results · up to 75.0 tok/s
- Tesla T102 results · up to 76.0 tok/s
- AMD Radeon RX 9060 XT2 results · up to 70.5 tok/s
- Intel Arc B5802 results · up to 72.0 tok/s
- AMD Radeon Pro W57002 results · up to 71.3 tok/s
- Intel Arc Pro B602 results · up to 70.5 tok/s
- AMD Radeon RX Vega 642 results · up to 67.7 tok/s
- AMD BC-2502 results · up to 63.1 tok/s
- GTX 1660 Ti Mobile2 results · up to 57.3 tok/s
- AMD Ryzen Al Max+ 3952 results · up to 53.6 tok/s
- Intel Arc A7702 results · up to 53.1 tok/s
- AMD FirePro S9300 x22 results · up to 55.6 tok/s
- AMD Radeon RX Vega 562 results · up to 52.7 tok/s
- Intel Arc B5702 results · up to 50.5 tok/s
- AMD Radeon RX 6800M2 results · up to 49.1 tok/s
- Intel Arc A770M2 results · up to 47.7 tok/s
- Intel Arc Pro B502 results · up to 47.7 tok/s
- P104-1002 results · up to 48.6 tok/s
- AMD Radeon Pro V3402 results · up to 47.5 tok/s
- Intel Arc A7502 results · up to 44.0 tok/s
- AMD Radeon RX 4702 results · up to 38.6 tok/s
- AMD Radeon RX 5702 results · up to 40.2 tok/s
- AMD Radeon RX 4802 results · up to 37.2 tok/s
- Apple M2 Ultra2 results · up to 199 tok/s
- GTX 9802 results · up to 34.2 tok/s
- AMD FirePro W81002 results · up to 29.5 tok/s
- Intel Core Ultra 200 Series2 results · up to 23.1 tok/s
- AMD Ryzen AI 9 300 Series2 results · up to 22.4 tok/s
- AMD Ryzen 6000 Series2 results · up to 21.3 tok/s
- Apple M2 Pro2 results · up to 21.0 tok/s
- AMD Ryzen 7000 Series2 results · up to 20.1 tok/s
- AMD Ryzen 8000 Series2 results · up to 21.3 tok/s
- AMD Radeon Pro WX 41002 results · up to 16.6 tok/s
- Apple M22 results · up to 13.5 tok/s
- Apple M12 results · up to 12.8 tok/s
- AMD Ryzen 5000 Series2 results · up to 11.0 tok/s
- Intel Core 1100 Series2 results · up to 10.6 tok/s
- AMD Ryzen 4000 Series2 results · up to 9.9 tok/s
- Tesla K402 results · up to 9.9 tok/s
- Intel Core Ultra 100 Series2 results · up to 8.2 tok/s
- Intel Core 8000 Series2 results · up to 3.4 tok/s
- Intel N1502 results · up to 2.9 tok/s
- RTX 50601 results · up to 185 tok/s
- RTX 5080 Laptop GPU1 results · up to 34.7 tok/s
- AMD Radeon 760M Graphics1 results · up to 10.4 tok/s
- AMD Radeon 740M Graphics1 results · up to 6.2 tok/s
- GTX 1650 SUPER1 results · up to 31.2 tok/s
- RTX 2000 Ada Generation Laptop GPU1 results · up to 29.5 tok/s
- A40-48Q1 results · up to 39.5 tok/s
- RTX 4090 Laptop GPU1 results · up to 207 tok/s
- Radeon RX 5500 XT1 results · up to 56.2 tok/s
- RTX 3050 Laptop GPU1 results · up to 4.8 tok/s
- GT 10301 results · up to 24.1 tok/s
- AMD Radeon RX 76001 results · up to 5.5 tok/s
- RTX A1000 Laptop GPU1 results · up to 123 tok/s
- AMD Instinct MI50/MI601 results · up to 12.9 tok/s
- RTX 500 Ada Generation Laptop GPU1 results · up to 81.6 tok/s
- Quadro P50001 results · up to 14.9 tok/s
- T1200 Laptop GPU1 results · up to 21.0 tok/s
- T550 Laptop GPU1 results · up to 50.6 tok/s
- RTX 20501 results · up to 47.9 tok/s
- MX1501 results · up to 13.5 tok/s
- Apple M3 Ultra1 results · up to 116 tok/s
- AMD Radeon Pro W6800X1 results · up to 86.5 tok/s
- AMD Radeon Instinct MI601 results · up to 78.2 tok/s
- AMD Radeon Pro V5401 results · up to 69.6 tok/s
- AMD Radeon Instinct MI251 results · up to 54.7 tok/s
- AMD Radeon RX 6600M1 results · up to 48.2 tok/s
- AMD Radeon RX 7600M XT1 results · up to 45.3 tok/s
- AMD Radeon RX 5801 results · up to 39.3 tok/s
- AMD Radeon Pro W55001 results · up to 36.8 tok/s
- AMD Radeon RX 6500 XT1 results · up to 27.8 tok/s
- Apple M31 results · up to 26.4 tok/s
- AMD FirePro S100001 results · up to 25.3 tok/s
- AMD Ryzen Z1 Extreme1 results · up to 18.8 tok/s
- AMD FirePro D7001 results · up to 16.6 tok/s
- AMD Radeon RX 5501 results · up to 10.2 tok/s
- MediaTek Dimensity 94001 results · up to 8.9 tok/s
- AMD Ryzen 3000 Series1 results · up to 8.5 tok/s
- Intel Core 1000 Series1 results · up to 4.3 tok/s
- CIX CD81801 results · up to 5.5 tok/s
- AMD Ryzen 5 3000 Series1 results · up to 8.5 tok/s
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.