Inference · fits in memory

Open models that fit in 8 GB.

Each model at its fastest build that fits in this much memory (on the card, or in RAM without one), leaving a fifth for the prompt. Sizes are what the build actually held while it ran, not its download.

68models fit
ModelBuild that fitsGB heldFastest tok/s
forge-1-mini · north-ml1Q2_K by north-ml1<0.11075
DeepSeek-OCR 2 BASEQ4_K_M by sabafallah5.2412
FunctionGemma 270MQ4_K_M by unsloth0.4341
DeepSeek-OCR BASEQ4_K_M by sabafallah4.7340
Gemma 3 270MQ4_K_M by unsloth1.0292
Gemma 4 E2BQ4_K_M by unsloth4.4291
Qwen2.5 0.5B InstructQ4_K_M by Qwen3.9248
Hunyuan MT 2 1.8BQ4_K_M by tencent2.8233
Llama 3.2 3BIQ1_S by unsloth1.5225
Phi 4 MINIQ4_K_M by maziyarpanahi5.8205
Sarvam-1Q4_K_M by bartowski5.4178
MiniCPM-V 4.6 BASEQ4_K_M by openbmb5.0144
HY-MT1.5-1.8B-i1-GGUFQ4_0 by mradermacher2.3132
Granite 4.0 SPEECH-1BQ4_K_M by ibm-granite1.5130
GLM-OCR BASEQ4_K_M by mradermacher2.2106
Llama 3.2 1BIQ1_S by maziyarpanahi0.6105
MiniCPM-V 4 BASEQ4_K_M by openbmb3.097
Gemma 4 E4BQ4_K_M by google3.294
Qwen 3 4BIQ1_S by unsloth1.890
Gemma 4 E2B · bartowskiQ4_0 by bartowski2.687
TranslateGemma 4BQ4_K_M by bullerwins2.978
MedGemma 1.5 4BQ4_K_M by unsloth3.675
Hunyuan MT 1.5 1.8BQ4_K_M by tencent2.346
functiongemma-270m-it-GGUFIQ4_NL by Mungert0.340
functiongemma-270m-it-unsloth-bnb-4bit-ft-QAT-GGUFQ4_K_M by mingxilei0.339
functiongemma-270m-it-abominable-heresyIQ4_XS by mradermacher0.339
functiongemma-270m-itQ4_K_M by victor0.337
Home-FunctionGemma-270mQ3_K_S by mradermacher0.336
Phi 4 MINI-REASONINGQ8_0 by lmstudio-community4.722
Qwen 3.5 0.8BQ4_K_M by unsloth0.817
LFM2.5 1.2BQ4_K_M by unsloth0.916
Qwen 3 0.6BQ8_0 by Qwen1.116
Gemma 3 1BQ4_K_M by ggml-org1.014
Granite 4.0 1BQ4_0 by ibm-granite1.412
Llama-3.2-1B-Instruct-Q4_K_M-GGUFQ4_K_M by hugging-quants1.011
Qwen 3 VL-2BQ4_K_M by Qwen2.09.3
embeddinggemma-300m-GGUF-with-dense-modulesQ6_K by cduk0.3—
EmbeddingGemma 2 BASEQ4_K_M by ngquocvinh0.4—
EmbeddingGemma 300MBF16 by unsloth0.7—
embeddinggemma-300m-ggufF16 by lm-kit0.7—
moonshine tinyspeech-pod by UsefulSensors0.7—
MMS-TTS BengaliFP32 by facebook1.2—
MMS-TTS TamilFP32 by facebook1.3—
MMS-TTS TeluguFP32 by facebook1.3—
MMS-TTS MarathiFP32 by facebook1.3—
MMS-TTS GujaratiFP32 by facebook1.4—
MMS-TTS KannadaFP32 by facebook1.6—
MMS-TTS MalayalamFP32 by facebook1.7—
MMS-TTS PunjabiFP32 by facebook1.7—
Whisper small telugu (vasista22)FP32 by vasista221.7—
MMS-TTS AssameseFP32 by facebook1.8—
MMS-TTS OdiaFP32 by facebook1.8—
MMS-TTS UrduFP32 by facebook1.8—
SAM ViT-BaseFP32 by facebook1.8—
kokoro basespeech-pod by kokoro1.9—
Whisper small hindi (vasista22)FP32 by vasista222.0—
Whisper small tamil (vasista22)FP32 by vasista222.0—
Whisper smallFP32 by openai2.6—
wespeaker-voxceleb-resnet34-LMFP32 by pyannote3.4—
spkrec-ecapa-voxcelebFP32 by speechbrain3.4—
Whisper mediumFP32 by openai3.4—
Whisper large-v3-turboFP16 by openai4.0—
XTTS-v2 (17 languages, 6-second clone)FP32 by coqui4.2—
Tiny SDFP32 by segmind4.8—
Seamless M4T v2 Large (speech → text)FP32 by facebook4.8—
Whisper small kannada (vasista22)FP32 by vasista225.0—
memorized-commons10diffusers by nbkcnil5.6—
SD-Turbo (SD 2.1 distilled)FP16 by stabilityai6.4—