Inference · fits in memory

Open models that fit in 4 GB.

Each model at its fastest build that fits in this much memory (on the card, or in RAM without one), leaving a fifth for the prompt. Sizes are what the build actually held while it ran, not its download.

51models fit
ModelBuild that fitsGB heldFastest tok/s
forge-1-mini · north-ml1Q2_K by north-ml1<0.11075
FunctionGemma 270MQ4_K_M by unsloth0.4341
Gemma 3 270MQ4_K_M by unsloth1.0292
Hunyuan MT 2 1.8BQ4_K_M by tencent2.8233
Llama 3.2 3BIQ1_S by unsloth1.5225
Gemma 4 E2BQ4_K_M by google1.3170
HY-MT1.5-1.8B-i1-GGUFQ4_0 by mradermacher2.3132
Granite 4.0 SPEECH-1BQ4_K_M by ibm-granite1.5130
GLM-OCR BASEQ4_K_M by mradermacher2.2106
Llama 3.2 1BIQ1_S by maziyarpanahi0.6105
MiniCPM-V 4 BASEQ4_K_M by openbmb3.097
Gemma 4 E4BQ4_K_M by google3.294
Qwen 3 4BIQ1_S by unsloth1.890
Gemma 4 E2B · bartowskiQ4_0 by bartowski2.687
TranslateGemma 4BQ4_K_M by bullerwins2.978
Phi 4 MINIQ2_K by unsloth2.371
Hunyuan MT 1.5 1.8BQ4_K_M by tencent2.346
functiongemma-270m-it-GGUFIQ4_NL by Mungert0.340
functiongemma-270m-it-unsloth-bnb-4bit-ft-QAT-GGUFQ4_K_M by mingxilei0.339
functiongemma-270m-it-abominable-heresyIQ4_XS by mradermacher0.339
functiongemma-270m-itQ4_K_M by victor0.337
Home-FunctionGemma-270mQ3_K_S by mradermacher0.336
Qwen 3.5 0.8BQ4_K_M by unsloth0.817
LFM2.5 1.2BQ4_K_M by unsloth0.916
Qwen 3 0.6BQ8_0 by Qwen1.116
Gemma 3 1BQ4_K_M by ggml-org1.014
Granite 4.0 1BQ4_0 by ibm-granite1.412
Llama-3.2-1B-Instruct-Q4_K_M-GGUFQ4_K_M by hugging-quants1.011
Qwen 3 VL-2BQ4_K_M by Qwen2.09.3
embeddinggemma-300m-GGUF-with-dense-modulesQ6_K by cduk0.3—
EmbeddingGemma 2 BASEQ4_K_M by ngquocvinh0.4—
EmbeddingGemma 300MBF16 by unsloth0.7—
embeddinggemma-300m-ggufF16 by lm-kit0.7—
moonshine tinyspeech-pod by UsefulSensors0.7—
MMS-TTS BengaliFP32 by facebook1.2—
MMS-TTS TamilFP32 by facebook1.3—
MMS-TTS TeluguFP32 by facebook1.3—
MMS-TTS MarathiFP32 by facebook1.3—
MMS-TTS GujaratiFP32 by facebook1.4—
MMS-TTS KannadaFP32 by facebook1.6—
MMS-TTS MalayalamFP32 by facebook1.7—
MMS-TTS PunjabiFP32 by facebook1.7—
Whisper small telugu (vasista22)FP32 by vasista221.7—
MMS-TTS AssameseFP32 by facebook1.8—
MMS-TTS OdiaFP32 by facebook1.8—
MMS-TTS UrduFP32 by facebook1.8—
SAM ViT-BaseFP32 by facebook1.8—
kokoro basespeech-pod by kokoro1.9—
Whisper small hindi (vasista22)FP32 by vasista222.0—
Whisper small tamil (vasista22)FP32 by vasista222.0—
Whisper smallFP32 by openai2.6—