Inference · hardware

What runs on Linux, 4 CPUs · 31 GB.

Every open model a machine of this kind has run on the network, at its fastest build, fastest first. Speed is tokens a second, measured and signed by the machine itself.

41models measured
1075fastest tok/s
3machines
ModelFastest build heretok/sContext$ / M tokens
Forge-1-Minibuild1075—$0.03
Qwen2.5 0.5B InstructQ4_K_M (4-bit) by Qwen23932K$0.56
Sarvam-1Q4_K_M (4-bit) by bartowski1114K$1.22
Llama 3.2 3BIQ1_S (1-bit) by unsloth94—$1.43
Gemma 4 E2BQ4_0 (4-bit) by unsloth84128K$1.63
Phi 4 MiniQ4_K_M (4-bit) by unsloth74128K$1.84
Qwen 3 4BIQ1_S (1-bit) by unsloth61—$2.21
FunctionGemma 270MIQ4_NL (4-bit) by Mungert40—$0.76
functiongemma-270m-it-unsloth-bnb-4bit-ft-QAT-GGUFQ4_K_M (4-bit) by mingxilei39—$0.78
functiongemma-270m-it-abominable-heresyIQ4_XS (4-bit) by mradermacher39—$0.79
Gemma 3 270MQ4_K_M (4-bit) by unsloth37—$0.82
Home-FunctionGemma-270mQ3_K_S (3-bit) by mradermacher36—$0.85
Airavata 7B (Hindi)Q4_K_M (4-bit) by mradermacher324K$4.36
OpenHathi 7B Hindi baseQ4_K_M (4-bit) by QuantFactory30—$4.62
Qwen 3.5 0.8BQ4_0 (4-bit) by unsloth17—$1.86
LFM2.5 1.2BQ4_K_M (4-bit) by unsloth164K$1.95
Sarvam-MQ4_K_M (4-bit) by bartowski168K$9.33
MiniCPM-V 4.7 1BQ8_0 (8-bit) by openbmb158K$2.09
Gemma 3 1BQ4_K_M (4-bit) by ggml-org148K$2.22
Krutrim-2 InstructQ4_K_M (4-bit) by bartowski1464K$10.35
Qwen 3 0.6BQ8_0 (8-bit) by Qwen134K$2.42
Llama 3.2 1BQ4_K_M (4-bit) by bartowski124K$2.54
Qwen 3 VL-4BF16 (16-bit, full precision) by Qwen12128K$11.83
Granite 4.0 1BQ4_K_M (4-bit) by ibm-granite114K$2.74
Qwen 3 VL-2BQ4_K_M (4-bit) by Qwen9.24K$3.38
Hunyuan MT 2 30B-A3BQ4_K_M (4-bit) by tencent5.44K$8.31
Huihui-Qwen3.6-35B-A3B-abliterated-MTP-GGUFQ2_K (2-bit) by huihui-ai5.0—$8.15
Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-PreservedQ4_K_M (4-bit) by llmfan464.84K$9.76
OpenAI-gpt-oss-20B-Claude-4.5-Opus-Heretic-UncensoredQ4_0 (4-bit) by mradermacher4.8—$8.26
Kimi Linear 48B-A3BQ4_K_M (4-bit) by bartowski4.84K$11.18
Kimi-Linear-Reap-35B-A3B-InstructQ4_K_M (4-bit) by mradermacher4.84K$9.88
gpt-oss 20BQ4_0 (4-bit) by unsloth4.64K$8.63
openai_gpt-oss-20b-GGUF-MXFP4-ExperimentalMXFP4 (4-bit) by bartowski4.3—$9.22
Qwen 3.5 35B-A3BQ4_K_S (4-bit) by unsloth4.04K$11.68
LFM2.5 24B-A2BIQ3_XXS (3-bit) by unsloth3.64K$10.27
GLM-4.7 FlashQ4_K_M (4-bit) by unsloth3.54K$12.57
gpt-oss-20b-DerestrictedQ5_K_M (5-bit) by MaziyarPanahi3.4—$12.66
gpt-oss-20b-hereticQ8_0 (8-bit) by DavidAU3.4—$14.15
Huihui-gpt-oss-20b-BF16-abliteratedQ8_0 (8-bit) by DavidAU3.3—$14.51
North 1.0 Mini-Code-1-0Q4_K_M (4-bit) by unsloth3.24K$14.16
Devstral-Small-2505-Deepseek-V3.2-Speciale-DistillIQ3_XS (3-bit) by TeichAI0.44K$105.21