Inference · fits in memory

Open models that fit in 12 GB.

Each model at its fastest build that fits in this much memory (on the card, or in RAM without one), leaving a fifth for the prompt. Sizes are what the build actually held while it ran, not its download.

81models fit
ModelBuild that fitsGB heldFastest tok/s
forge-1-mini · north-ml1Q2_K by north-ml1<0.11075
DeepSeek-OCR 2 BASEQ4_K_M by sabafallah5.2412
MiniCPM5 1BQ4_K_M by openbmb7.4381
FunctionGemma 270MQ4_K_M by unsloth0.4341
DeepSeek-OCR BASEQ4_K_M by sabafallah4.7340
Gemma 3 270MQ4_K_M by unsloth1.0292
Gemma 4 E2BQ4_K_M by unsloth4.4291
Qwen2.5 0.5B InstructQ4_K_M by Qwen3.9248
Hunyuan MT 2 1.8BQ4_K_M by tencent2.8233
Llama 3.2 3BIQ1_S by unsloth1.5225
Phi 4 MINIQ4_K_M by maziyarpanahi5.8205
Sarvam-1Q4_K_M by bartowski5.4178
MiniCPM-V 4.6 BASEQ4_K_M by openbmb5.0144
Qwen 3 4BQ4_K_M by Qwen9.0141
HY-MT1.5-1.8B-i1-GGUFQ4_0 by mradermacher2.3132
Granite 4.0 SPEECH-1BQ4_K_M by ibm-granite1.5130
GLM-OCR BASEQ4_K_M by mradermacher2.2106
Llama 3.2 1BIQ1_S by maziyarpanahi0.6105
MiniCPM-V 4 BASEQ4_K_M by openbmb3.097
Gemma 4 E4BQ4_K_M by google3.294
Gemma 4 E2B · bartowskiQ4_0 by bartowski2.687
TranslateGemma 4BQ4_K_M by bullerwins2.978
SmolLM3 3BQ4_K_M by bartowski7.378
MedGemma 1.5 4BQ4_K_M by unsloth3.675
Hunyuan MT 1.5 1.8BQ4_K_M by tencent2.346
MiniCPM4.1 8BQ4_K_M by openbmb7.746
functiongemma-270m-it-GGUFIQ4_NL by Mungert0.340
functiongemma-270m-it-unsloth-bnb-4bit-ft-QAT-GGUFQ4_K_M by mingxilei0.339
functiongemma-270m-it-abominable-heresyIQ4_XS by mradermacher0.339
functiongemma-270m-itQ4_K_M by victor0.337
Home-FunctionGemma-270mQ3_K_S by mradermacher0.336
Gemma 4 12B · yuxinlu1Q4_K_M by yuxinlu18.026
Gemma 4 12BIQ4_XS by unsloth7.223
Phi 4 MINI-REASONINGQ8_0 by lmstudio-community4.722
MiniCPM-o 4.5 BASEQ4_K_M by openbmb9.720
Qwen 3.5 0.8BQ4_K_M by unsloth0.817
LFM2.5 1.2BQ4_K_M by unsloth0.916
Qwen 3 0.6BQ8_0 by Qwen1.116
Gemma 3 1BQ4_K_M by ggml-org1.014
Granite 4.0 1BQ4_0 by ibm-granite1.412
Llama-3.2-1B-Instruct-Q4_K_M-GGUFQ4_K_M by hugging-quants1.011
Qwen 3 VL-2BQ4_K_M by Qwen2.09.3
LFM2.5 24B-A2BIQ3_XXS by unsloth9.13.6
Ministral 3 14B-INSTRUCT-2512Q4_K_M by mistralai9.11.8
embeddinggemma-300m-GGUF-with-dense-modulesQ6_K by cduk0.3—
EmbeddingGemma 2 BASEQ4_K_M by ngquocvinh0.4—
EmbeddingGemma 300MBF16 by unsloth0.7—
embeddinggemma-300m-ggufF16 by lm-kit0.7—
moonshine tinyspeech-pod by UsefulSensors0.7—
MMS-TTS BengaliFP32 by facebook1.2—
MMS-TTS TamilFP32 by facebook1.3—
MMS-TTS TeluguFP32 by facebook1.3—
MMS-TTS MarathiFP32 by facebook1.3—
MMS-TTS GujaratiFP32 by facebook1.4—
MMS-TTS KannadaFP32 by facebook1.6—
MMS-TTS MalayalamFP32 by facebook1.7—
MMS-TTS PunjabiFP32 by facebook1.7—
Whisper small telugu (vasista22)FP32 by vasista221.7—
MMS-TTS AssameseFP32 by facebook1.8—
MMS-TTS OdiaFP32 by facebook1.8—
MMS-TTS UrduFP32 by facebook1.8—
SAM ViT-BaseFP32 by facebook1.8—
kokoro basespeech-pod by kokoro1.9—
Whisper small hindi (vasista22)FP32 by vasista222.0—
Whisper small tamil (vasista22)FP32 by vasista222.0—
Whisper smallFP32 by openai2.6—
wespeaker-voxceleb-resnet34-LMFP32 by pyannote3.4—
spkrec-ecapa-voxcelebFP32 by speechbrain3.4—
Whisper mediumFP32 by openai3.4—
Whisper large-v3-turboFP16 by openai4.0—
XTTS-v2 (17 languages, 6-second clone)FP32 by coqui4.2—
Tiny SDFP32 by segmind4.8—
Seamless M4T v2 Large (speech → text)FP32 by facebook4.8—
Whisper small kannada (vasista22)FP32 by vasista225.0—
memorized-commons10diffusers by nbkcnil5.6—
SD-Turbo (SD 2.1 distilled)FP16 by stabilityai6.4—
baseFP16 by Darylgeero6.9—
Depth Anything V2 BaseFP32 by depth-anything7.8—
Depth Anything V2 SmallFP32 by depth-anything7.8—
MMS-TTS HindiFP32 by facebook8.8—
phi-lattice-v2 · ununununununununununununununununununungeneration by ununununununununununununununununununun9.0—