A build is one file: a packager’s quantization of one model, run by one engine. Each row is measured on real machines and signed by the machine that measured it. Speeds are tokens a second; the fastest names its machine, the typical is the middle of every machine that ran it.
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 4.4 | 291RTX PRO 6000 Blackwell · 96 GB | 54 | 13 | 128K | 20 + 37 lang |
| IQ4_XS by unsloth · ollama | 3.4 | 183GeForce RTX 3080 Ti · 12 GB | 79 | 3 | 128K | 21 + 29 lang |
| Q4_K_M by google · ollama | 1.3 | 170Windows, 16 CPUs · 16 GB | 19 | 10 | 128K | 20 + 37 lang |
| Q4_0 by unsloth · ollama | 2.6 | 85 | 58 | 3 | 128K | 21 + 10 lang |
| Q8_0 by unsloth · ollama | 5.7 | 67 | 5.3 | 2 | 128K | 21 + 22 lang |
| IQ4_NL by unsloth · ollama | — | 55 | 32 | 2 | 8K | 21 + 14 lang |
| Q3_K_M by unsloth · ollama | — | 38 | 14 | 3 | 8K | 19 + 10 lang |
| BF16 by unsloth · ollama | 15 | 38 | 3.1 | 2 | 128K | 21 + 20 lang |
| Q3_K_S by unsloth · ollama | — | 14 | — | 1 | 4K | 19 + 14 lang |
| ollama by ollama · ollama | — | 8.9 | — | 1 | — | 0 |
| llamacpp by google · llamacpp | — | 2.0 | — | 1 | — | 1 |
| Q2_K_XL by unsloth · llamacpp | — | not timed | — | 1 | — | 10 + 6 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by google · ollama | 3.2 | 94 | — | 1 | 128K | 21 + 14 lang |
| IQ4_XS by unsloth · ollama | 5.9 | 61 | — | 1 | 128K | 21 + 10 lang |
| Q4_K_M by unsloth · ollama | 6.2 | 57 | — | 1 | 128K | 20 + 10 lang |
| Q4_0 by unsloth · ollama | 6.0 | 55 | — | 1 | 128K | 21 + 9 lang |
| Q8_0 by unsloth · ollama | 8.3 | 39 | — | 1 | 128K | 19 + 14 lang |
| BF16 by unsloth · ollama | 11 | 26 | — | 1 | 128K | 21 + 13 lang |
| Q3_K_M by unsloth · ollama | — | 20 | — | 1 | 4K | 21 + 9 lang |
| ollama by ollama · ollama | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_0 by bartowski · ollama | 2.6 | 87 | — | 1 | 128K | 21 + 10 lang |
| Q4_K_M by bartowski · ollama | 2.7 | 73 | — | 1 | 128K | 21 + 10 lang |
| Q3_K_L by bartowski · ollama | 2.5 | 73 | — | 1 | 128K | 21 + 10 lang |
| IQ2_M by bartowski · ollama | 2.2 | 71 | — | 1 | 128K | 20 + 10 lang |
| IQ3_XS by bartowski · ollama | 2.3 | 62 | — | 1 | 128K | 21 + 10 lang |
| Q8_0 by bartowski · ollama | 3.6 | 61 | — | 1 | 128K | 21 + 10 lang |
| BF16 by bartowski · ollama | 15 | 28 | — | 1 | 128K | 21 + 10 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q8_0 by unsloth · ollama | 23 | 21 | 3.0 | 2 | 128K | 21 + 21 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by maziyarpanahi · ollama | 5.8 | 205GeForce RTX 3080 Ti · 12 GB | — | 1 | 4K | 6 + 5 lang |
| Q2_K by maziyarpanahi · ollama | — | 154Windows, 16 CPUs · 16 GB | — | 1 | — | 4 + 1 lang |
| Q3_K_S by maziyarpanahi · ollama | — | 135Windows, 16 CPUs · 16 GB | 28 | 3 | 4K | 6 + 1 lang |
| Q3_K_L by maziyarpanahi · ollama | 22 | 110Windows, 16 CPUs · 16 GB | 64 | 3 | 128K | 5 + 1 lang |
| Q4_K_M by unsloth · ollama | 22 | 82 | 3.1 | 5 | 128K | 6 + 2 lang |
| Q4_K_S by maziyarpanahi · ollama | — | 77Windows, 16 CPUs · 16 GB | — | 1 | — | 6 + 2 lang |
| Q2_K by unsloth · ollama | 2.3 | 71 | — | 1 | — | 4 + 1 lang |
| Q5_K_M by unsloth · ollama | 23 | 69 | — | 1 | 128K | 6 + 4 lang |
| Q3_K_M by unsloth · ollama | 22 | 67Linux, 4 CPUs · 31 GB | — | 1 | 128K | 4 + 2 lang |
| Q8_0 by unsloth · ollama | 24 | 54 | — | 1 | 128K | 6 + 2 lang |
| Q4_K_M by bartowski · ollama | 3.1 | 41 | — | 1 | 4K | 6 + 2 lang |
| Q8_0 by maziyarpanahi · ollama | — | 39Windows, 16 CPUs · 16 GB | — | 1 | — | 5 + 2 lang |
| IQ2_M by bartowski · ollama | 2.1 | 37 | 14 | 2 | 4K | 5 + 1 lang |
| IQ4_XS by bartowski · ollama | 2.8 | 36Mac, 10 CPUs · 16 GB | — | 1 | 4K | 5 + 2 lang |
| Q4_0 by bartowski · ollama | 2.9 | 34Mac, 10 CPUs · 16 GB | — | 1 | 4K | 6 + 1 lang |
| Q3_K_L by bartowski · ollama | — | 32 | — | 1 | 8K | 6 + 3 lang |
| BF16 by unsloth · ollama | 26 | 32 | 0.1 | 2 | 128K | 5 + 3 lang |
| Q3_K_S by bartowski · ollama | — | 28Mac, 10 CPUs · 16 GB | — | 1 | 8K | 5 + 2 lang |
| IQ2_XS by mungert · ollama | — | 8.5 | — | 1 | — | 4 + 1 lang |
| IQ3_XXS by bartowski · ollama | 2.3 | 0.7Mac, 10 CPUs · 16 GB | — | 1 | 4K | 6 + 1 lang |
| BF16 by microsoft · vllm | — | not timed | — | 1 | 4K | 6 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ3_XXS by unsloth · ollama | 10 | 189Windows, 16 CPUs · 16 GB | 34 | 6 | 4K | 7 + 2 lang |
| IQ1_S by unsloth · ollama | — | 189 | 5.6 | 9 | — | 0 + 1 lang |
| Q3_K_L by lmstudio-community · ollama | — | 90Windows, 16 CPUs · 16 GB | 14 | 4 | — | 7 + 1 lang |
| Q2_K by unsloth · ollama | — | 38 | 24 | 3 | — | 7 + 1 lang |
| Q3_K_M by unsloth · ollama | — | 32 | 20 | 4 | — | 7 + 1 lang |
| Q8_0 by lmstudio-community · ollama | 4.7 | 22Mac, 10 CPUs · 16 GB | — | 1 | 4K | 7 + 2 lang |
| Q4_K_M by lmstudio-community · ollama | — | 7.3Mac, 8 CPUs · 8 GB | 6.0 | 2 | — | 7 + 1 lang |
| Q2_K_L by unsloth · ollama | — | 6.8 | 6.8 | 3 | — | 5 |
| Q4_K_M by unsloth · ollama | — | 6.5Windows, 8 CPUs · 16 GB | 4.2 | 2 | — | 6 |
| Q6_K by lmstudio-community · ollama | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 0.4 | 341L4 · 23 GB | 34 | 3 | — | 4 + 3 lang |
| IQ2_XXS by unsloth · ollama | 0.3 | 67Intel(R) Iris(R) Xe Graphics · 1 GB | 50 | 2 | — | 2 |
| Q4_0 by unsloth · ollama | 0.3 | 37Linux, 4 CPUs · 31 GB | — | 1 | — | 4 + 1 lang |
| Q8_0 by unsloth · ollama | 0.3 | 33Linux, 4 CPUs · 31 GB | — | 1 | — | 2 + 2 lang |
| IQ3_XXS by unsloth · ollama | 0.2 | 33Linux, 4 CPUs · 31 GB | — | 1 | — | 2 |
| IQ2_M by unsloth · ollama | 0.2 | 33Linux, 4 CPUs · 31 GB | 18 | 2 | — | 2 |
| BF16 by unsloth · ollama | 0.6 | 19Linux, 4 CPUs · 31 GB | — | 1 | — | 2 + 3 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ4_NL by Mungert · ollama | 0.3 | 40 | — | 1 | — | 2 + 2 lang |
| Q3_K_S by Mungert · ollama | 0.2 | 34 | — | 1 | — | 2 + 1 lang |
| Q4_1 by Mungert · ollama | 0.2 | 34 | — | 1 | — | 2 + 3 lang |
| Q4_K_M by Mungert · ollama | 0.3 | 33 | — | 1 | — | 2 + 2 lang |
| Q8_0 by Mungert · ollama | 0.5 | 22 | — | 1 | — | 2 + 2 lang |
| BF16 by Mungert · ollama | 0.6 | 20 | — | 1 | — | 2 + 3 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ4_XS by mradermacher · ollama | 0.3 | 39Linux, 4 CPUs · 31 GB | — | 1 | — | 2 + 2 lang |
| Q3_K_S by mradermacher · ollama | 0.3 | 37 | — | 1 | — | 2 |
| Q2_K by mradermacher · ollama | 0.3 | 35Linux, 4 CPUs · 31 GB | — | 1 | — | 2 |
| Q4_K_M by mradermacher · ollama | 0.3 | 34Linux, 4 CPUs · 31 GB | — | 1 | — | 3 + 1 lang |
| Q4_K_S by mradermacher · ollama | 0.3 | 34Linux, 4 CPUs · 31 GB | — | 1 | — | 3 + 1 lang |
| Q8_0 by mradermacher · ollama | 0.3 | 32Linux, 4 CPUs · 31 GB | — | 1 | — | 2 + 1 lang |
| F16 by mradermacher · ollama | 0.6 | 23Linux, 4 CPUs · 31 GB | — | 1 | — | 2 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by victor · ollama | 0.3 | 37 | — | 1 | — | 4 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q3_K_S by mradermacher · ollama | 0.3 | 36Linux, 4 CPUs · 31 GB | — | 1 | — | 0 |
| IQ4_XS by mradermacher · ollama | 0.3 | 35Linux, 4 CPUs · 31 GB | — | 1 | — | 0 |
| Q2_K by mradermacher · ollama | 0.3 | 35Linux, 4 CPUs · 31 GB | — | 1 | — | 0 |
| Q4_K_M by mradermacher · ollama | 0.3 | 33Linux, 4 CPUs · 31 GB | — | 1 | — | 0 |
| Q4_K_S by mradermacher · ollama | 0.3 | 33Linux, 4 CPUs · 31 GB | — | 1 | — | 0 |
| Q8_0 by mradermacher · ollama | 0.3 | 32Linux, 4 CPUs · 31 GB | — | 1 | — | 0 |
| F16 by mradermacher · ollama | 0.6 | 23Linux, 4 CPUs · 31 GB | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by Qwen · ollama | 9.0 | 141 | 50 | 3 | 40K | 14 + 4 lang |
| IQ1_S by unsloth · ollama | 1.8 | 90Windows, 16 CPUs · 16 GB | 61 | 3 | — | 0 |
| BF16 by unsloth · ollama | 11 | 81Windows, 16 CPUs · 16 GB | — | 1 | 16K | 14 + 2 lang |
| IQ3_XXS by unsloth · ollama | 12 | 63 | 26 | 4 | 32K | 14 + 2 lang |
| Q5_0 by Qwen · ollama | 7.9 | 49 | 23 | 3 | 32K | 14 + 3 lang |
| Q5_K_M by Qwen · ollama | 7.9 | 47Linux, 4 CPUs · 31 GB | 41 | 2 | 32K | 16 + 2 lang |
| Q4_0 by unsloth · ollama | — | 41 | 17 | 4 | 4K | 15 + 6 lang |
| Q8_0 by Qwen · ollama | 9.3 | 39Linux, 4 CPUs · 31 GB | 28 | 2 | 32K | 14 + 1 lang |
| Q4_1 by unsloth · ollama | — | 37 | 20 | 2 | 4K | 15 + 6 lang |
| Q4_1 by unsloth | — | 19 | 16 | 2 | — | 0 |
| Q2_K by unsloth · ollama | — | 12 | — | 1 | — | 15 + 4 lang |
| IQ4_XS by unsloth · ollama | — | 6.4 | — | 1 | 4K | 12 + 6 lang |
| vllm by qwen · vllm | — | not timed | — | 1 | 64K | 12 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 29 | 27 | — | 1 | 128K | 6 + 9 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q2_K by mradermacher · ollama | — | 22 | 2.6 | 3 | 4K | 12 + 4 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| F16 by Qwen · ollama | 30 | 12Linux, 4 CPUs · 31 GB | — | 1 | 128K | 14 + 6 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by Qwen · ollama | 2.0 | 9.3Linux, 4 CPUs · 31 GB | — | 1 | 4K | 13 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| F16 by Qwen · ollama | 32 | 3.8 | — | 1 | 128K | 14 + 7 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ1_S by unsloth · ollama | 1.5 | 225Windows, 16 CPUs · 16 GB | 160 | 2 | — | 0 + 1 lang |
| Q4_0 by unsloth · ollama | — | 205Windows, 16 CPUs · 16 GB | — | 1 | — | 10 + 1 lang |
| Q4_K_M by unsloth · ollama | 17 | 204Windows, 16 CPUs · 16 GB | 97 | 3 | 128K | 12 + 7 lang |
| IQ3_XXS by unsloth · ollama | — | 200Windows, 16 CPUs · 16 GB | — | 1 | — | 11 + 4 lang |
| IQ2_XXS by unsloth · ollama | — | 124Windows, 16 CPUs · 16 GB | — | 1 | — | 5 + 1 lang |
| Q4_K_M by bartowski · ollama | 22 | 99GeForce RTX 3080 Ti · 12 GB | 93 | 2 | 128K | 12 + 5 lang |
| F16 by unsloth · ollama | 8.8 | 95Windows, 16 CPUs · 16 GB | — | 1 | 16K | 11 + 5 lang |
| Q3_K_L by bartowski · ollama | 22 | 79Linux, 4 CPUs · 31 GB | — | 1 | 128K | 12 + 2 lang |
| Q8_0 by unsloth · ollama | — | 73Windows, 16 CPUs · 16 GB | — | 1 | — | 12 + 3 lang |
| IQ3_M by bartowski · ollama | 22 | 72Linux, 4 CPUs · 31 GB | — | 1 | 128K | 12 + 3 lang |
| Q8_0 by bartowski · ollama | 24 | 63 | — | 1 | 128K | 10 + 3 lang |
| IQ1_S by maziyarpanahi · ollama | — | 46 | — | 1 | — | 3 |
| F16 by bartowski · ollama | 24 | 38 | — | 1 | 128K | 11 + 3 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by maziyarpanahi · ollama | — | 113Mac, 10 CPUs · 16 GB | 61 | 2 | 8K | 11 + 2 lang |
| IQ3_XS by maziyarpanahi · ollama | — | 108 | 18 | 3 | 4K | 10 + 4 lang |
| IQ1_S by maziyarpanahi · ollama | 0.6 | 105 | 8.6 | 4 | — | 0 |
| Q4_K_M by bartowski · ollama | 1.0 | 104 | 12 | 4 | 16K | 12 + 6 lang |
| Q8_0 by ollama · ollama | 2.6 | 50 | 9.6 | 16 | 32K | 11 + 7 lang |
| Q3_K_L by bartowski · ollama | 0.9 | 10Linux, 4 CPUs · 31 GB | — | 1 | — | 9 + 3 lang |
| Q8_0 by bartowski · ollama | 1.5 | 8.7Linux, 4 CPUs · 31 GB | — | 1 | 4K | 11 + 2 lang |
| F16 by bartowski · ollama | 2.7 | 8.0 | 5.5 | 2 | — | 11 + 5 lang |
| IQ3_M by bartowski · ollama | 0.9 | 5.3Linux, 4 CPUs · 31 GB | — | 1 | — | 11 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ3_XS by bartowski · ollama | — | 47machine not reported | — | 1 | 8K | 9 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by hugging-quants · ollama | 1.0 | 11 | — | 1 | — | 3 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| BF16 by unsloth · ollama | 0.7 | not timed | — | 1 | — | 1 |
| F32 by unsloth · ollama | 1.3 | not timed | — | 2 | — | 1 |
| Q2_K by gaianet · ollama | 0.3 | not timed | — | 1 | — | 1 |
| Q2_K by second-state · ollama | 0.3 | not timed | — | 1 | — | 1 |
| Q3_K_S by gaianet · ollama | 0.3 | not timed | — | 1 | — | 1 |
| Q3_K_S by second-state · ollama | 0.3 | not timed | — | 1 | — | 1 |
| Q4_0 by unsloth · ollama | 0.3 | not timed | — | 1 | — | 1 |
| Q4_K_M by second-state · ollama | 0.3 | not timed | — | 4 | — | 1 |
| Q4_K_M by gaianet · ollama | 0.3 | not timed | — | 3 | — | 1 |
| Q8_0 by ggml-org · ollama | 0.4 | not timed | — | 1 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 1.0 | 292L4 · 23 GB | 46 | 6 | — | 4 + 3 lang |
| IQ2_M by unsloth · ollama | 0.2 | 69Intel(R) Iris(R) Xe Graphics · 1 GB | 52 | 2 | — | 3 + 1 lang |
| IQ3_XXS by unsloth · ollama | 0.3 | 52Intel(R) Iris(R) Xe Graphics · 1 GB | 43 | 2 | — | 3 + 1 lang |
| IQ2_XXS by unsloth · ollama | 0.3 | 35Intel(R) Iris(R) Xe Graphics · 1 GB | 35 | 2 | — | 2 |
| Q4_0 by unsloth · ollama | 0.3 | 33Linux, 4 CPUs · 31 GB | — | 1 | — | 4 + 1 lang |
| Q8_0 by unsloth · ollama | 0.4 | 31Linux, 4 CPUs · 31 GB | — | 1 | 32K | 3 + 1 lang |
| F16 by unsloth · ollama | 0.6 | 24Linux, 4 CPUs · 31 GB | — | 1 | 16K | 3 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by ggml-org · ollama | 1.0 | 14Linux, 4 CPUs · 31 GB | 13 | 2 | 8K | 5 + 7 lang |
| Q3_K_S by maziyarpanahi · ollama | 0.8 | 14Linux, 4 CPUs · 31 GB | — | 1 | 8K | 5 + 1 lang |
| Q3_K_L by lmstudio-community · ollama | 0.9 | 13Linux, 4 CPUs · 31 GB | — | 1 | 8K | 5 + 5 lang |
| Q2_K by maziyarpanahi · ollama | 0.8 | 13Linux, 4 CPUs · 31 GB | — | 1 | 8K | 5 + 1 lang |
| Q6_K by lmstudio-community · ollama | 1.2 | 11Linux, 4 CPUs · 31 GB | — | 1 | 8K | 5 + 5 lang |
| Q8_0 by ggml-org · ollama | 1.2 | 9.7Linux, 4 CPUs · 31 GB | — | 1 | 8K | 5 + 6 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 15 | 73 | 14 | 6 | 128K | 16 + 21 lang |
| Q3_K_S by unsloth · ollama | 15 | 552× Tesla T4 · 15 GB | 4.4 | 3 | 64K | 14 + 19 lang |
| Q8_K_XL by unsloth · ollama | 18 | 442× Tesla T4 · 15 GB | 24 | 2 | 128K | 16 + 10 lang |
| Q4_0 by unsloth · ollama | 12 | 4.6Linux, 4 CPUs · 31 GB | 4.2 | 3 | 4K | 14 + 17 lang |
| Q2_K by unsloth · ollama | 12 | 4.3Linux, 4 CPUs · 31 GB | 4.2 | 2 | 4K | 16 + 18 lang |
| Q8_0 by unsloth · ollama | 12 | 4.0Linux, 4 CPUs · 31 GB | 4.0 | 2 | 4K | 15 + 13 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| MXFP4 by bartowski · ollama | 12 | 4.3Linux, 4 CPUs · 31 GB | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q8_0 by DavidAU · ollama | 22 | 3.9 | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q5_K_M by MaziyarPanahi · ollama | 17 | 3.4Linux, 4 CPUs · 31 GB | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q8_0 by DavidAU · ollama | 22 | 3.3Linux, 4 CPUs · 31 GB | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 12 | 42L4 · 23 GB | 23 | 2 | 256K | 15 + 9 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.8 | 5.8s of audio in 5.8s · 1.0× slower than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.2 | 5.3s of audio in 5.4s · 1.0× slower than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.4 | 6.1s of audio in 6.1s · 1.0× slower than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 8.8 | 5.5s of audio in 6.8s · 1.2× slower than realtime · on Windows, 8 cores, 16 GB | — | 5 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.6 | 6.1s of audio in 5.9s · 1.0× faster than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.7 | 4.0s of audio in 4.5s · 1.1× slower than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.3 | 7.4s of audio in 7.1s · 1.0× faster than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.8 | 5.3s of audio in 5.2s · 1.0× faster than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.7 | 6.8s of audio in 6.4s · 1.1× faster than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.3 | 9.1s of audio in 8.9s · 1.0× faster than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.3 | 6.4s of audio in 6.1s · 1.1× faster than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 1.8 | 9.0s of audio in 18s · 2.0× slower than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 27 | 9.5 | 3.2 | 4 | 128K | 16 + 9 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ1_M by unsloth · ollama | — | not timed | — | 1 | 4K | 14 + 9 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by peculiar-ragdoll · ollama | 26 | 10machine not reported | — | 1 | 128K | 21 + 10 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q6_K_P by HauhauCS · ollama | 31 | 1.8machine not reported | — | 1 | 64K | 11 + 10 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by ibm-granite · ollama | 1.5 | 1302× Tesla T4 · 15 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_0 by ibm-granite · ollama | 1.4 | 12 | — | 1 | 4K | 9 + 1 lang |
| Q4_K_M by ibm-granite · ollama | 1.4 | 11Linux, 4 CPUs · 31 GB | 11 | 2 | 4K | 9 + 5 lang |
| Q3_K_S by ibm-granite · ollama | 1.2 | 8.5 | — | 1 | 4K | 7 + 1 lang |
| Q2_K by ibm-granite · ollama | 1.1 | 8.2Linux, 4 CPUs · 31 GB | — | 1 | — | 2 + 1 lang |
| Q3_K_L by ibm-granite · ollama | 1.3 | 7.2Linux, 4 CPUs · 31 GB | — | 1 | 4K | 9 + 1 lang |
| BF16 by ibm-granite · ollama | 3.7 | 4.7Linux, 4 CPUs · 31 GB | — | 1 | 4K | 7 + 4 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by mradermacher · ollama | 2.2 | 106L4 · 23 GB | 60 | 2 | 4K | 7 + 1 lang |
| Q3_K_S by mradermacher · ollama | 2.1 | 18Intel(R) Iris(R) Xe Graphics · 1 GB | — | 1 | 4K | 6 + 1 lang |
| Q2_K by mradermacher · ollama | 2.0 | 13Intel(R) Iris(R) Xe Graphics · 1 GB | — | 1 | — | 6 + 1 lang |
| FP8 by axteam · vllm | — | not timed | — | 1 | 4K | 10 + 1 lang |
| W4A16 by AlphaOxO · vllm | — | not timed | — | 1 | 4K | 5 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP16 by Darylgeero · diffusers | 6.9 | 512×512 in 49s · 0.02 steps/sec · on Linux, 4 cores, 31 GB | — | 1 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| diffusers by nbkcnil · diffusers | 5.6 | 512×512 in 37s · 0.03 steps/sec · on Linux, 4 cores, 31 GB | — | 1 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| generation by ununununununununununununununununununun · generation | 10 | not timed | — | 3 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| generation by ununununununununununununununununununun · generation | 9.0 | not timed | — | 2 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP16 by stabilityai · diffusers | 6.4 | 512×512 in 53s · 0.02 steps/sec · on Linux, 4 cores, 31 GB | — | 7 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| diffusers by zai-org · diffusers | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by tencent · ollama | 19 | 5.4Linux, 4 CPUs · 31 GB | 4.5 | 2 | 4K | 7 + 10 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 20 | 3.2Linux, 4 CPUs · 31 GB | 2.7 | 2 | 4K | 11 + 9 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ3_XXS by mradermacher · ollama | 12 | 2.3 | — | 1 | 4K | 11 + 9 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP16 by openai · speech-hf | — | 21s of audio in 15s · 1.4× faster than realtime · on Linux, 12 cores, 53 GB | — | 1 | — | 2 + 10 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP16 by openai · speech-hf | 4.0 | 21s of audio in 3m 23s · 9.7× slower than realtime · on Linux, 4 cores, 31 GB | — | 6 | — | 2 + 10 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by openai · speech-hf | 3.4 | 16s of audio in 3m 41s · 14.0× slower than realtime · on Windows, 8 cores, 16 GB | — | 2 | — | 2 + 4 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by openai · speech-hf | 2.6 | 19s of audio in 2m 29s · 7.9× slower than realtime · on Windows, 8 cores, 16 GB | — | 3 | — | 2 + 6 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by vasista22 · speech-hf | 2.0 | 19s of audio in 16m 51s · 53.6× slower than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by vasista22 · speech-hf | 5.0 | 6.4s of audio in 4m 40s · 43.6× slower than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by vasista22 · speech-hf | 2.0 | 23s of audio in 19m 40s · 52.5× slower than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by vasista22 · speech-hf | 1.7 | 20s of audio in 37m 7s · 112.8× slower than realtime · on Linux, 4 cores, 31 GB | — | 1 | — | 2 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by ibm-granite · ollama | 13 | 17L4 · 23 GB | 3.5 | 3 | 32K | 16 + 3 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by ibm-granite · ollama | — | 1.9machine not reported | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by ibm-granite · ollama | — | 0.7 | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by bartowski · ollama | 30 | 4.8Linux, 4 CPUs · 31 GB | 3.9 | 2 | 4K | 14 + 9 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by mradermacher · ollama | 21 | 4.7Linux, 4 CPUs · 31 GB | — | 1 | 4K | 6 + 7 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by mradermacher · ollama | — | 3.4 | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 0.9 | 16Linux, 4 CPUs · 31 GB | — | 1 | 4K | 12 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q8_0 by DavidAU · ollama | — | 4.1 | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ3_XXS by unsloth · ollama | 9.1 | 3.6Linux, 4 CPUs · 31 GB | — | 1 | 4K | 16 + 9 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| generation by lightx2v · generation | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| generation by prunaai · generation | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| generation by optimum-intel-internal-testing · generation | — | not timed | — | 2 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| BF16 by Wan-AI · diffusers | — | not timed | — | 1 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by depth-anything · vision-task | 7.8 | 230×86 in 28s · on Windows, 8 cores, 16 GB | — | 8 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by depth-anything · vision-task | 7.8 | 230×86 in 1.1s · on Windows, 8 cores, 16 GB | — | 8 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| BF16 by yujiepan · vllm | — | not timed | — | 1 | — | 0 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| BF16 by yujiepan · vllm | — | not timed | — | 1 | — | 0 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by SPRINGLab · speech-hf | 11 | 18s of audio in 9m 7s · 31.0× slower than realtime · on Windows, 8 cores, 16 GB | — | 5 | — | 2 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by SWivid · speech-hf | 11 | 6.8s of audio in 22s · 3.2× slower than realtime · on Linux, 12 cores, 53 GB | — | 1 | — | 2 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_0 by mradermacher · ollama | 2.3 | 132 | 132 | 2 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by tencent · ollama | 2.3 | 46L4 · 23 GB | 27 | 2 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| build by lightricks | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| BF16 by Lightricks · diffusers | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by mistralai · ollama | 31 | 17L4 · 23 GB | 4.7 | 3 | 64K | 10 + 2 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by mistralai · ollama | 9.1 | 1.8 | — | 1 | 4K | 13 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| speech-pod by UsefulSensors · speech-pod | — | 16s of audio in 3.4s · 4.7× faster than realtime · on Linux, 4 cores, 31 GB | — | 5 | — | 2 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| speech-pod by UsefulSensors · speech-pod | 0.7 | 16s of audio in 1.7s · 9.4× faster than realtime · on Linux, 4 cores, 31 GB | — | 6 | — | 2 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · audiogen | 15 | not timed | — | 2 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · audiogen | 12 | 4.0s of audio in 1m 56s · 28.9× slower than realtime · on Windows, 8 cores, 16 GB | — | 11 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · translate-hf | 15 | not timed | — | 3 | — | 13 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · translate-hf | 13 | not timed | — | 8 | — | 13 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| generation by beh1222 · generation | 14 | 512×512 in 6.6s · 0.15 steps/sec · on Linux, 12 cores, 53 GB | — | 1 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP16 by Efficient-Large-Model · diffusers | 13 | not timed | — | 4 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| build by languaman | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| build by aemotionstudio | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by mradermacher · ollama | 13 | 53 | 19 | 3 | 4K | 3 + 2 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ3_XS by TeichAI · ollama | 11 | 0.4Linux, 4 CPUs · 31 GB | — | 1 | 4K | 6 + 6 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by sabafallah · ollama | 4.7 | 340L4 · 23 GB | 177 | 2 | — | 3 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by sabafallah · ollama | 5.2 | 412GeForce RTX 3080 Ti · 12 GB | 335 | 3 | — | 4 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by speechbrain · speaker-id | 3.4 | not timed | — | 3 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by ngquocvinh · ollama | 0.4 | not timed | — | 1 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| BF16 by nm-testing · vllm | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 28 | 213 | 2.2 | 14 | 128K | 16 + 16 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by mradermacher · ollama | — | 7.7 | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 24 | 18L4 · 23 GB | — | 1 | 64K | 10 + 3 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| BF16 by yujiepan · vllm | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| BF16 by hunyuanvideo-community · diffusers | — | not timed | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ3_XXS by mradermacher · ollama | — | 0.4 | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| speech-pod by kokoro · speech-pod | 1.9 | 4.2s of audio in 2.4s · 1.7× faster than realtime · on Windows, 8 cores, 16 GB | — | 6 | — | 2 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by bartowski · ollama | 20 | 15L4 · 23 GB | 14 | 3 | 64K | 6 + 10 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by google · translate-hf | 14 | not timed | — | 2 | — | 11 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 21 | 5.5L4 · 23 GB | — | 1 | 40K | 11 + 10 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by unsloth · ollama | 3.6 | 75L4 · 23 GB | — | 1 | 64K | 9 + 9 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by openbmb · ollama | 3.0 | 97L4 · 23 GB | 3.7 | 3 | — | 10 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by openbmb · ollama | 15 | 11L4 · 23 GB | — | 1 | 16K | 18 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by openbmb · ollama | 7.4 | 381GeForce RTX 3080 Ti · 12 GB | 115 | 4 | 64K | 15 + 1 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ1_S by mradermacher · ollama | — | 1.0 | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| IQ1_S by mradermacher · ollama | — | 1.4 | — | 1 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by meta-models · ollama | 15 | 6.9L4 · 23 GB | 1.3 | 3 | — | 0 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by QuantFactory · ollama | 13 | 52 | 30 | 2 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by Reza2kn · ollama | 23 | 18L4 · 23 GB | — | 1 | 32K | 14 + 6 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| BF16 by sarvamai · translate-hf | 20 | not timed | — | 2 | — | 14 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · speech-hf | 4.8 | 12s of audio in 2m 8s · 11.1× slower than realtime · on Windows, 8 cores, 16 GB | — | 3 | — | 2 + 13 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by facebook · vision-task | 1.8 | 5 frames at 230×86 in 12m 7s · on Windows, 8 cores, 16 GB | — | 9 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by bartowski · ollama | 7.3 | 78L4 · 23 GB | — | 1 | 64K | 6 + 2 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| Q4_K_M by bullerwins · ollama | 2.9 | 78L4 · 23 GB | 41 | 2 | 128K | 5 + 7 lang |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by pyannote · speaker-id | 3.4 | not timed | — | 3 | — | 1 |
| Build | GB | Fastest tok/s | Typical | Machines | Context | Checks |
|---|---|---|---|---|---|---|
| FP32 by coqui · speech-hf | 4.2 | 6.2s of audio in 28s · 4.5× slower than realtime · on Linux, 12 cores, 53 GB | — | 1 | — | 2 |