Inference · fits in memory

Open models that fit in 80 GB.

Each model at its fastest build that fits in this much memory (on the card, or in RAM without one), leaving a fifth for the prompt. Sizes are what the build actually held while it ran, not its download.

135models fit
ModelBuild that fitsGB heldFastest tok/s
forge-1-mini · north-ml1Q2_K by north-ml1<0.11075
DeepSeek-OCR 2 BASEQ4_K_M by sabafallah5.2412
MiniCPM5 1BQ4_K_M by openbmb7.4381
FunctionGemma 270MQ4_K_M by unsloth0.4341
DeepSeek-OCR BASEQ4_K_M by sabafallah4.7340
Gemma 3 270MQ4_K_M by unsloth1.0292
Gemma 4 E2BQ4_K_M by unsloth4.4291
Qwen2.5 0.5B InstructQ4_K_M by Qwen3.9248
Hunyuan MT 2 1.8BQ4_K_M by tencent2.8233
Llama 3.2 3BIQ1_S by unsloth1.5225
MiniCPM-V 4.7 1BQ4_K_M by openbmb14221
GLM-4.7 FLASHQ4_K_M by unsloth28213
Phi 4 MINIQ4_K_M by maziyarpanahi5.8205
Phi 4 MINI-REASONINGIQ3_XXS by unsloth10189
Sarvam-1Q4_K_M by bartowski5.4178
MiniCPM-V 4.6 BASEQ4_K_M by openbmb5.0144
Qwen 3 4BQ4_K_M by Qwen9.0141
HY-MT1.5-1.8B-i1-GGUFQ4_0 by mradermacher2.3132
Granite 4.0 SPEECH-1BQ4_K_M by ibm-granite1.5130
GLM-OCR BASEQ4_K_M by mradermacher2.2106
Llama 3.2 1BIQ1_S by maziyarpanahi0.6105
MiniCPM-V 4 BASEQ4_K_M by openbmb3.097
Gemma 4 E4BQ4_K_M by google3.294
Gemma 4 E2B · bartowskiQ4_0 by bartowski2.687
TranslateGemma 4BQ4_K_M by bullerwins2.978
SmolLM3 3BQ4_K_M by bartowski7.378
MedGemma 1.5 4BQ4_K_M by unsloth3.675
gpt-oss 20BQ4_K_M by unsloth1573
Airavata 7B (Hindi)Q4_K_M by mradermacher1353
OpenHathi 7B Hindi baseQ4_K_M by QuantFactory1352
Hunyuan MT 1.5 1.8BQ4_K_M by tencent2.346
MiniCPM4.1 8BQ4_K_M by openbmb7.746
Qwen 3.5 4BQ4_K_M by unsloth1242
functiongemma-270m-it-GGUFIQ4_NL by Mungert0.340
functiongemma-270m-it-unsloth-bnb-4bit-ft-QAT-GGUFQ4_K_M by mingxilei0.339
functiongemma-270m-it-abominable-heresyIQ4_XS by mradermacher0.339
functiongemma-270m-itQ4_K_M by victor0.337
Home-FunctionGemma-270mQ3_K_S by mradermacher0.336
Gemma 4 12BQ4_K_M by unsloth1429
30b-a3b · unslothQ4_K_M by unsloth2927
Gemma 4 12B · yuxinlu1Q4_K_M by yuxinlu18.026
Phi 4 REASONING-PLUSQ4_K_M by unsloth1725
Gemma 4 26B-A4BQ8_0 by unsloth2321
MiniCPM-o 4.5 BASEQ4_K_M by openbmb9.720
Granite 4.1 3BQ4_K_M by unsloth2418
Qianfan-OCR BASEQ4_K_M by Reza2kn2318
Qwen 3.5 0.8BQ4_K_M by unsloth0.817
Granite 4.2 3BQ4_K_M by ibm-granite1317
Ministral 3 3B-INSTRUCT-2512Q4_K_M by mistralai3117
LFM2.5 1.2BQ4_K_M by unsloth0.916
Qwen 3 0.6BQ8_0 by Qwen1.116
Sarvam-MQ4_K_M by bartowski1716
Qwen 3 14BQ8_0 by Qwen2216
Krutrim-2 InstructQ4_K_M by bartowski2015
Gemma 3 1BQ4_K_M by ggml-org1.014
Huihui-Qwen3.8-27B-abliterated-GGUFQ4_K_S by huihui-ai2513
Qwen 3 VL-4BF16 by Qwen3012
Granite 4.0 1BQ4_0 by ibm-granite1.412
MiniCPM-V 4.5 BASEQ4_K_M by openbmb1511
Llama-3.2-1B-Instruct-Q4_K_M-GGUFQ4_K_M by hugging-quants1.011
Dirk-Qwen3.8-27B-GGUFQ4_K_M by peculiar-ragdoll2610
Qwen3.8-27B-Uncensored-GGUFQ4_K_M by JonathanColetti2610
Qwen 3.6 27BQ4_K_M by unsloth279.5
Qwen 3 VL-2BQ4_K_M by Qwen2.09.3
Muse Glimmer 30BQ4_K_M by meta-models156.9
Magistral Small 2506Q4_K_M by unsloth215.5
Hunyuan MT 2 30B-A3BQ4_K_M by tencent195.4
Huihui-Qwen3.6-35B-A3B-abliterated-MTP-GGUFQ2_K by huihui-ai135.0
Qwen3.6-35B-A3B-uncensored-heretic-Native-MTP-PreservedQ4_K_M by llmfan46224.9
OpenAI-gpt-oss-20B-Claude-4.5-Opus-Heretic-UncensoredQ4_0 by mradermacher124.8
Kimi Linear 48B-A3BQ4_K_M by bartowski304.8
Kimi-Linear-REAP-35B-A3B-InstructQ4_K_M by mradermacher214.7
openai_gpt-oss-20b-GGUF-MXFP4-ExperimentalMXFP4 by bartowski124.3
Qwen 3.5 35B-A3BQ4_K_M by unsloth224.2
gpt-oss-20b-hereticQ8_0 by DavidAU223.9
Qwen 3 VL-8BF16 by Qwen323.8
LFM2.5 24B-A2BIQ3_XXS by unsloth9.13.6
gpt-oss-20b-DerestrictedQ5_K_M by MaziyarPanahi173.4
Gemma 4 31BQ4_K_M by unsloth203.4
Huihui-gpt-oss-20b-BF16-abliteratedQ8_0 by DavidAU223.3
North 1.0 MINI-CODE-1-0Q4_K_M by unsloth203.2
Qwen 3.8 27BQ8_0 by unsloth322.9
North-Mini-Code-1.0-i1-GGUFIQ3_XXS by mradermacher122.3
Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUFQ6_K_P by HauhauCS311.8
Ministral 3 14B-INSTRUCT-2512Q4_K_M by mistralai9.11.8
Devstral-Small-2505-Deepseek-V3.2-Speciale-DistillIQ3_XS by TeichAI110.4
embeddinggemma-300m-GGUF-with-dense-modulesQ6_K by cduk0.3—
EmbeddingGemma 2 BASEQ4_K_M by ngquocvinh0.4—
EmbeddingGemma 300MBF16 by unsloth0.7—
embeddinggemma-300m-ggufF16 by lm-kit0.7—
moonshine tinyspeech-pod by UsefulSensors0.7—
MMS-TTS BengaliFP32 by facebook1.2—
MMS-TTS TamilFP32 by facebook1.3—
MMS-TTS TeluguFP32 by facebook1.3—
MMS-TTS MarathiFP32 by facebook1.3—
MMS-TTS GujaratiFP32 by facebook1.4—
MMS-TTS KannadaFP32 by facebook1.6—
MMS-TTS MalayalamFP32 by facebook1.7—
MMS-TTS PunjabiFP32 by facebook1.7—
Whisper small telugu (vasista22)FP32 by vasista221.7—
MMS-TTS AssameseFP32 by facebook1.8—
MMS-TTS OdiaFP32 by facebook1.8—
MMS-TTS UrduFP32 by facebook1.8—
SAM ViT-BaseFP32 by facebook1.8—
kokoro basespeech-pod by kokoro1.9—
Whisper small hindi (vasista22)FP32 by vasista222.0—
Whisper small tamil (vasista22)FP32 by vasista222.0—
Whisper smallFP32 by openai2.6—
wespeaker-voxceleb-resnet34-LMFP32 by pyannote3.4—
spkrec-ecapa-voxcelebFP32 by speechbrain3.4—
Whisper mediumFP32 by openai3.4—
Whisper large-v3-turboFP16 by openai4.0—
XTTS-v2 (17 languages, 6-second clone)FP32 by coqui4.2—
Tiny SDFP32 by segmind4.8—
Seamless M4T v2 Large (speech → text)FP32 by facebook4.8—
Whisper small kannada (vasista22)FP32 by vasista225.0—
memorized-commons10diffusers by nbkcnil5.6—
SD-Turbo (SD 2.1 distilled)FP16 by stabilityai6.4—
baseFP16 by Darylgeero6.9—
Depth Anything V2 BaseFP32 by depth-anything7.8—
Depth Anything V2 SmallFP32 by depth-anything7.8—
MMS-TTS HindiFP32 by facebook8.8—
phi-lattice-v2 · ununununununununununununununununununungeneration by ununununununununununununununununununun9.0—
phi-lattice · ununununununununununununununununununungeneration by ununununununununununununununununununun10—
F5-TTS v1 Base (zero-shot voice, en/zh)FP32 by SWivid11—
F5-Hindi 24 kHz (SPRING Lab)FP32 by SPRINGLab11—
MusicGen SmallFP32 by facebook12—
NLLB-200 distilled 600MFP32 by facebook13—
Sana 1.6B 1024pxFP16 by Efficient-Large-Model13—
600m · beh1222generation by beh122214—
MADLAD-400 3B MTFP32 by google14—
MusicGen MediumFP32 by facebook15—
NLLB-200 distilled 1.3BFP32 by facebook15—
CogVideoX 2BBF16 by THUDM17—
Sarvam-Translate (Gemma 3 4B)BF16 by sarvamai20—