Hardware /
RTX 4050 Laptop GPU
234measured results
155best gen tok/s*
6 GBreported memory
*Headline excludes tiny/test models (under 0.6B params or TinyStories-style). Open in rig checker →
llama.cpp
| Model | Quant | Gen tok/s | Prompt tok/s | Source | Context tested | Build | |
|---|---|---|---|---|---|---|---|
| Sailor-0.5B-Chat | Q4_K - Medium | 155 | 9,049 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2.5 Coder 0.5B Instruct | Q6_K | 142 | 9,277 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD OLMo 1B SFT DPO | Q4_K - Medium | 139 | 6,524 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| LamoFast_2.0 | Q8_0 | 134 | 9,362 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 132 | 5,332 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 132 | 5,231 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Qwen3-0.6B | Q4_K - Medium | 122 | 6,559 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| LightOnOCR 2 1B | Q4_K - Medium | 119 | 6,786 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Beck 0.6B | Q4_K - Medium | 118 | 6,774 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| ReasonLite 0.6B Turbo | Q4_K - Medium | 117 | 6,679 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| InternVL3_5 1B | Q4_K - Medium | 116 | 6,705 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Theta Crucis 0.6B Turbo1 | Q5_K - Medium | 115 | 6,533 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen3 0.6B | Q4_K - Medium | 115 | 6,639 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 114 | 4,979 | LocalScore ↗ | 512+128 | llamafile 0.9.2 | Details → |
| ReasonLite 0.6B | Q4_K - Medium | 114 | 6,885 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Astral 0.6B Coder | Q5_K - Medium | 113 | 6,593 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Deepseek Coder 1.3b Instruct | Q4_K - Medium | 107 | 5,212 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| MinerU HTML | Q4_K - Medium | 106 | 6,591 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DeepBrainz R1 0.6B v2 | Q4_K - Medium | 106 | 6,629 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Titulm Mpt 1b v2.0 | Q4_K - Medium | 102 | 4,778 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gemma 3 1b It Qat | Q4_0 | 100 | 8,463 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| tinyllama_tinyllama-1.1b-chat-v1.0 | Q8_0 | 99.5 | 3,987 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gemma-3-1B-It | Q4_K - Medium | 98.1 | 8,031 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Nusantara 0.8b Indo Chat | Q4_K - Medium | 94.2 | 5,652 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 1B Instruct | Q4_K - Medium | 94.1 | 4,127 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Refact 1_6B Fim | Q4_K - Medium | 92.9 | 3,189 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| ordis_v355_VarGH_merged | Q4_K - Medium | 92.8 | 4,816 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen3 1.7B | Q4_K - Medium | 91.0 | 3,512 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| DeepSeek R1 Distill Qwen 1.5B | Q4_K - Medium | 90.3 | 4,795 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gemma 3 1b It | Q4_K - Medium | 87.0 | 8,007 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| VibeThinker 1.5B | Q5_K - Medium | 86.1 | 4,694 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| OpenCoder 1.5B Instruct | Q4_K - Medium | 82.1 | 4,476 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| OpenCoder 1.5B Instruct | Q4_K - Medium | 81.2 | 4,249 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen3 Coder 0.6B 0.9B 7 1bR5 Nobs F16 | Q4_K - Medium | 79.0 | 4,600 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Sailor2 1B Chat | Q4_K - Medium | 78.5 | 4,527 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Superthoughts Lite v2 MOE Llama3.2 | Q4_K - Medium | 78.2 | 2,911 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Kakaocorp.Kanana 1.5 2.1b Instruct 2505 | Q4_K - Medium | 73.1 | 3,037 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| EXAONE 3.5 2.4B Instruct | Q4_K - Medium | 72.2 | 2,756 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Neo 3 3B A400M Base | Q4_K - Medium | 68.4 | 2,324 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Fijik1.5 2.6b A380M | Q4_K - Medium | 66.2 | 2,472 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TowerVision 2B | Q4_K - Medium | 65.4 | 3,533 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Granite 3.1 1b A400M Instruct | Q5_K - Medium | 62.1 | 3,457 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 3.2 3B Instruct | Q4_K - Medium | 60.0 | 2,518 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Starcoder2 3b | Q4_K - Medium | 59.0 | 2,377 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Nanbeige4.1 3B | Q4_K - Medium | 56.7 | 2,332 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Nanbeige4 3B Thinking 2511 | Q4_K - Medium | 55.8 | 2,387 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Phi 4 Mini Instruct | Q4_K - Medium | 50.5 | 2,065 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Phi-4-Mini-Reasoning | Q4_K - Medium | 50.1 | 2,066 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| EuroMoE 2.6B A0.6B Instruct Preview | Q4_K - Medium | 49.0 | 1,635 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gemma-3-4B-It | Q4_K - Medium | 46.7 | 2,551 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Rio 3.0 Open Mini | Q4_K - Medium | 45.8 | 1,785 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen3-4B-Instruct-2507 | Q4_K - Medium | 45.6 | 1,804 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gemma 3 4b It | Q4_K - Medium | 45.5 | 2,455 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen3 4B Thinking 2507 | Q4_K - Medium | 45.3 | 1,806 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 2 7B | Q4_0 | 43.7 | 1,726 | llama.cpp (CUDA) ↗ | 512+128 | llama.cpp CUDA @d79d8f3 | Details → |
| Syngen Reasoning 0.6b | BF16 | 42.5 | 3,838 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 2 7B | Q4_0 | 41.9 | 1,154 | llama.cpp (Vulkan) ↗ | 512+128 | llama.cpp Vulkan @d79d8f3 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 31.7 | 1,217 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Meta Llama 3.1 8B Instruct | Q4_K - Medium | 31.0 | 1,185 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Nemotron Cascade 8B | Q4_K - Medium | 30.1 | 1,132 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| KomdigiUB 8B Instruct DTP | Q4_K - Medium | 29.9 | 1,151 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DeepSeek R1 0528 Qwen3 8B | Q4_K - Medium | 29.7 | 1,159 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Falcon3 Mamba 7B Instruct | Q4_0 | 14.7 | 82.6 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
171 tiny/test model result(s) (excluded from headlines)
| Model | Quant | Gen tok/s | Prompt tok/s | Source | Context tested | Build | |
|---|---|---|---|---|---|---|---|
| Tiny Llama Ultra Compact | Q4_K - Medium | 3,091 | 220,409 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Planck-OpenLAiNN-10M | F16 | 2,811 | 218,864 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Planck-OpenLAiNN-10M | Q4_0 | 2,330 | 206,117 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Gpt2 | Q4_K - Medium | 2,032 | 235,917 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q4_K - Medium | 1,905 | 145,343 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q4_K - Medium | 1,890 | 144,854 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q5_1 | 1,890 | 145,629 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q5_K - Small | 1,890 | 142,236 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q4_1 | 1,887 | 148,849 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Raincandy U TinyStories 656K | Q4_K - Medium | 1,885 | 153,940 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q4_K - Small | 1,885 | 140,884 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q5_K - Medium | 1,879 | 153,054 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q6_K | 1,873 | 159,348 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q5_K - Medium | 1,869 | 145,396 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q4_0 | 1,859 | 147,634 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q5_0 | 1,856 | 154,382 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Raincandy U TinyStories 656K | Q8_0 | 1,823 | 153,794 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories-656K | Q4_K - Medium | 1,732 | 153,201 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Random Phi3ForCausalLM | Q4_K - Medium | 1,686 | 175,578 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| llama-39m | Q4_K - Medium | 1,529 | 124,862 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Random Olmo Hf | Q4_K - Medium | 1,492 | 175,963 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny_Random_Llama2_New | Q4_K - Medium | 1,492 | 167,051 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Mpt Random Remote Code | Q3_K - Medium | 1,420 | 52,177 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Falcon | Q4_K - Medium | 1,414 | 101,933 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| vicuna-68m | Q4_K - Medium | 1,375 | 90,490 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Random Orion | Q4_K - Medium | 1,169 | 158,772 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Random Qwen3 | Q4_K - Medium | 958 | 167,212 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| llama | all F32 (guessed) | 908 | 92,104 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Bloom | all F32 | 899 | 135,614 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q3_K - Medium | 898 | 38,782 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Random Qwen3 | Q4_K - Medium | 897 | 157,670 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q3_K - Large | 865 | 39,062 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q3_K - Medium | 854 | 32,354 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Gemma Test | Q4_K - Medium | 849 | 129,655 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| llama | Q6_K | 846 | 79,711 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| llama | Q4_K - Medium | 844 | 87,067 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Random Internlm2 | Q4_K - Medium | 833 | 124,180 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Experimental Neo TinyStories Korean 800K 20240819 | Q4_K - Medium | 824 | 74,343 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Lm 8M | Q4_K - Medium | 807 | 74,142 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | IQ3_S mix - 3.66 bpw | 728 | 21,230 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tinyllama 15M | Q4_K - Medium | 718 | 74,446 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| NexTrim 50M | Q4_K - Medium | 704 | 59,232 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Pythia 14m | Q4_K - Medium | 697 | 101,949 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories-656K | IQ4_XS - 4.25 bpw | 674 | 15,970 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tinyllama 15M | Q4_K - Medium | 673 | 77,040 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyLLama | F16 | 657 | 38,404 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | IQ4_NL - 4.5 bpw | 651 | 25,663 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q2_K - Medium | 646 | 21,761 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Pythia 14m | Q4_K - Medium | 643 | 101,046 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Raincandy U TinyStories 656K | Q2_K - Medium | 640 | 27,724 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Granite 3.1 2b Instruct | Q6_K | 639 | 96,076 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| GPTNeoX Spanish_Poet 70m | Q4_K - Medium | 628 | 53,522 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | IQ4_XS - 4.25 bpw | 625 | 30,058 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | IQ3_S - 3.4375 bpw | 617 | 23,654 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | IQ3_XS - 3.3 bpw | 617 | 24,298 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Pythia 70m Deduped | Q4_K - Medium | 615 | 59,284 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Distilgpt2 Tiny Conversational | Q6_K | 605 | 40,369 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyStories 656K | Q3_K - Small | 557 | 14,647 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyLLama v0 | Q4_K - Medium | 528 | 34,967 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DAC60M | Q4_K - Medium | 525 | 45,270 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q4_0 | 522 | 34,113 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q5_0 | 521 | 32,770 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyLLama v0 | Q4_K - Medium | 519 | 33,597 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q5_1 | 517 | 30,545 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q4_K - Small | 517 | 32,351 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q4_K - Medium | 513 | 32,676 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q4_1 | 510 | 33,013 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q8_0 | 509 | 33,036 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 04 | Q4_K - Medium | 505 | 31,669 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 02 | Q4_K - Medium | 504 | 30,373 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q6_K | 502 | 32,163 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q5_K - Medium | 502 | 31,486 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q4_K - Medium | 502 | 33,450 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Plyx 15M | Q4_K - Medium | 486 | 54,473 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q4_K - Medium | 478 | 32,105 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| mistral-60m | Q4_K - Medium | 476 | 49,439 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Config.Json | Q8_0 | 469 | 95,601 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Lite-Oute-1-65M-Instruct | Q4_K - Medium | 469 | 29,126 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q4_0 | 459 | 33,267 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Pico-OpenLAiNN-100M | F16 | 448 | 36,744 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Phi 3 Mini 4k Instruct | Q4_K - Medium | 414 | 57,990 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Mistral | Q4_K - Medium | 409 | 17,695 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Tiny Random Stablelm 2 | Q4_K - Medium | 407 | 29,877 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Granite 3.0 1b A400M Instruct | Q6_K | 400 | 64,106 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Llama 135m | Q4_K - Medium | 379 | 55,309 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| AMD Llama 135m | Q4_K - Medium | 378 | 52,402 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Stentor 12M Instruct | Q4_K - Medium | 363 | 55,775 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Mamba Mistral Instruct Distill | Q4_K - Medium | 349 | 21,636 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Out_Bak_100M_2K_Code_Iter_160000 | Q4_K - Medium | 337 | 23,097 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Amd Pretrained 135m 2k Slimpajama_No_Book3 | Q4_K - Medium | 323 | 22,291 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Pythia 160m Deduped | Q4_K - Medium | 323 | 22,273 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gpt2 | Q4_K - Medium | 301 | 21,300 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| LocutusqueXFelladrin TinyMistral248M Instruct | Q6_K | 299 | 10,880 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Jeeves Small 100M | Q4_K - Medium | 290 | 23,345 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| TinyMistral 248M v2.5 | Q6_K | 286 | 11,023 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Rain | Q4_0 | 282 | 22,390 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| MiniPLM Qwen 200M | Q6_K | 280 | 21,889 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| llama | IQ4_XS - 4.25 bpw | 279 | 13,888 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llama 2 Tiny 4kv Heads 16layers Random | Q4_K - Medium | 260 | 10,922 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | BF16 | 239 | 23,089 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Qwen2 96M | Q4_K - Medium | 234 | 19,873 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Jais Family 256m Chat | Q4_K - Medium | 230 | 11,486 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| OpenELM 270M Instruct | Q4_K - Medium | 220 | 12,968 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q4_K - Medium | 219 | 10,670 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q2_K - Medium | 211 | 9,347 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q2_K - Small | 210 | 8,760 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q4_1 | 206 | 10,482 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q5_0 | 204 | 10,493 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q4_0 | 204 | 10,661 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q3_K - Small | 204 | 9,703 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q4_K - Small | 203 | 10,347 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q5_K - Small | 202 | 10,384 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q3_K - Medium | 201 | 10,136 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q5_K - Medium | 200 | 10,035 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q4_K - Medium | 198 | 10,530 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q3_K - Large | 196 | 10,154 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q5_1 | 196 | 10,424 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q6_K | 189 | 9,569 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| OpenELM 270M | Q4_K - Medium | 187 | 12,828 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| OuteTTS 0.1 350M Hf | Q4_K - Medium | 187 | 10,662 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | unknown, may not work | 175 | 10,676 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q3_K - Medium | 175 | 1,976 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | Q8_0 | 173 | 9,982 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q3_K - Large | 172 | 2,096 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Llm 350m Instruct v2 | Q4_K - Medium | 169 | 9,257 | LocalScore ↗ | 4,352 | llamafile 0.9.3 | Details → |
| Gemma 3 270m Qat Q4_0 Unquantized | Q4_0 | 168 | 23,746 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Functiongemma-270M-It | Q4_K - Medium | 166 | 23,516 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DogeAI v1.5 Coder | Q4_K - Medium | 163 | 23,266 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gemma 3 270m It Qat Q4_0 Unquantized | Q4_0 | 163 | 24,023 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gemma-3-270M-It | Q4_K - Medium | 159 | 23,440 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| MobileLLM 80M Finetuned | Q6_K | 153 | 14,090 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| MobileLLM 125M HF | Q4_K - Medium | 153 | 13,033 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| SmolLM2 135M Instruct | Q4_0 | 152 | 13,164 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| SmollerLM-48M-Instruct-ft-sft | Q4_K - Medium | 146 | 14,471 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| SmolVLM2 256M Video Instruct | Q4_K - Medium | 146 | 12,890 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| SmolLM2 135M Instruct | Q4_K - Medium | 145 | 13,269 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DMindAI.DMind 3 Nano | Q4_K - Medium | 143 | 21,525 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Distil Home Assistant Functiongemma | F16 | 139 | 22,376 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gemma 3 270m It | Q4_K - Medium | 138 | 23,291 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | F16 | 136 | 9,907 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| SmolDocling 256M Preview | Q4_K - Medium | 135 | 13,054 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| SmolLM2 135M Instruct | Q4_K - Medium | 129 | 13,112 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | IQ4_NL - 4.5 bpw | 128 | 1,546 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| OpenCLAW SEED 135M | Q4_K - Medium | 127 | 13,254 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | IQ4_XS - 4.25 bpw | 125 | 1,483 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q3_K - Small | 125 | 1,431 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Minueza 2 96M Instruct Variant 03 | Q2_K - Medium | 123 | 1,464 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| MTLM1 200M | Q4_K - Medium | 122 | 8,946 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| SmolVLM2 500M Video Instruct | Q4_K - Medium | 121 | 7,382 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | all F32 | 80.3 | 6,559 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| mamba-130m-hf | Q4_K - Medium | 79.9 | 881 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Monad | Q4_K - Medium | 75.9 | 12,204 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| MiniPLM Mamba 130M | Q4_K - Medium | 75.8 | 1,053 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Mamba 130m Hf | Q4_K - Small | 74.2 | 824 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| Gemma-3-270M-It-Qat | IQ4_XS - 4.25 bpw | 59.3 | 1,111 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | BF16 | 47.8 | 5,258 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| DistilGPT OSS Qwen3 4B | Q4_K - Medium | 45.2 | 1,763 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ1_S - 1.5625 bpw | 38.6 | 150 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| State Spaces Mamba 370m Hf | Q4_K - Medium | 38.1 | 400 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| SmolLM2 135M Instruct | IQ1_S - 1.5625 bpw | 35.4 | 739 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| SmolLM2 135M Instruct | IQ4_XS - 4.25 bpw | 35.0 | 860 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ2_XS - 2.3125 bpw | 34.9 | 143 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ2_M - 2.7 bpw | 30.5 | 135 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ1_M - 1.75 bpw | 30.4 | 94.5 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ3_S mix - 3.66 bpw | 30.3 | 122 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ2_S - 2.5 bpw | 30.1 | 131 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ2_XXS - 2.0625 bpw | 28.9 | 138 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ4_XS - 4.25 bpw | 28.4 | 329 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ3_XXS - 3.0625 bpw | 26.3 | 120 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ4_NL - 4.5 bpw | 26.3 | 259 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
| h2o-danube3-500m-chat | IQ3_S - 3.4375 bpw | 24.9 | 94.5 | LocalScore ↗ | 512+128 | llamafile 0.9.3 | Details → |
Ollama
No measured Ollama results yet. Submit yours →
MLX
No measured MLX results yet. Submit yours →
Includes results from LocalScore, a Mozilla Builders project by cjpais, mirrored in the cn0303/local-llm-speed-benchmarks dataset. LocalScore runs use llamafile, which is built on llama.cpp.