Inference · hardware · Google Colab

What runs on L4 · 23 GB.

Every open model a machine of this kind has run on the network, at its fastest build, fastest first. Speed is tokens a second, measured and signed by the machine itself.

33models measured
341fastest tok/s
1machine
ModelFastest build heretok/sContext$ / M tokens
FunctionGemma 270MQ4_K_M (4-bit) by unsloth341—$0.63
DeepSeek-OCR BaseQ4_K_M (4-bit) by sabafallah340—$0.64
DeepSeek-OCR 2 BaseQ4_K_M (4-bit) by sabafallah335—$0.65
Gemma 3 270MQ4_K_M (4-bit) by unsloth292—$0.74
MiniCPM-V 4.7 1BQ4_K_M (4-bit) by openbmb221256K$0.98
MiniCPM5 1BQ4_K_M (4-bit) by openbmb20864K$1.04
Gemma 4 E2BQ4_K_M (4-bit) by google150128K$1.48
MiniCPM-V 4.6 BaseQ4_K_M (4-bit) by openbmb131256K$1.65
Sarvam-1Q4_K_M (4-bit) by bartowski1218K$1.79
GLM-OCR BaseQ4_K_M (4-bit) by mradermacher1064K$2.04
Llama 3.2 3BQ4_K_M (4-bit) by unsloth97128K$2.23
MiniCPM-V 4 BaseQ4_K_M (4-bit) by openbmb97—$2.25
Phi 4 MiniQ4_K_M (4-bit) by unsloth79128K$2.74
SmolLM3 3BQ4_K_M (4-bit) by bartowski7864K$2.80
TranslateGemma 4BQ4_K_M (4-bit) by bullerwins78128K$2.81
MedGemma 1.5 4BQ4_K_M (4-bit) by unsloth7564K$2.92
Qwen 3 4BQ4_K_M (4-bit) by Qwen5040K$4.37
Hunyuan MT 1.5 1.8BQ4_K_M (4-bit) by tencent46—$4.66
MiniCPM4.1 8BQ4_K_M (4-bit) by openbmb4664K$4.79
Qwen 3.5 4BQ4_K_M (4-bit) by unsloth42256K$5.21
GLM-4.7 FlashQ4_K_M (4-bit) by unsloth41128K$5.67
Hunyuan MT 2 1.8BQ4_K_M (4-bit) by tencent37—$5.93
gpt-oss 20BQ4_K_M (4-bit) by unsloth24128K$9.35
MiniCPM-o 4.5 BaseQ4_K_M (4-bit) by openbmb208K$10.85
Granite 4.1 3BQ4_K_M (4-bit) by unsloth1864K$11.97
Qianfan-OCR BaseQ4_K_M (4-bit) by Reza2kn1832K$12.40
Granite 4.2 3BQ4_K_M (4-bit) by ibm-granite1732K$12.89
Ministral 3 3B-Instruct-2512Q4_K_M (4-bit) by mistralai1764K$13.16
Krutrim-2 InstructQ4_K_M (4-bit) by bartowski1432K$15.28
MiniCPM-V 4.5 BaseQ4_K_M (4-bit) by openbmb1116K$19.40
Muse Glimmer 30BQ4_K_M (4-bit) by meta-models6.9—$33.14
Qwen 3.6 27BQ4_K_M (4-bit) by unsloth5.5128K$41.38
Magistral Small 2506Q4_K_M (4-bit) by unsloth5.540K$41.40