Side by side only on machines of the same kind, so the difference is the model, not the computer. Each figure is the fastest build of that model on that machine.
| Machine | Gemma 4 E2B tok/s | Qwen2.5 0.5B Instruct tok/s | Context A / B |
|---|---|---|---|
| Linux, 4 CPUs · 31 GB | 84Q4_0 (4-bit) by unsloth | 239Q4_K_M (4-bit) by Qwen | 128K / 32K |
| GeForce RTX 3080 Ti · 12 GB | 183IQ4_XS (4-bit) by unsloth | 248Q4_K_M (4-bit) by Qwen | 4K / 8K |
| Linux PC with an NVIDIA GPU, 32 GB | 78Q4_K_M (4-bit) by unsloth | 239Q4_K_M (4-bit) by Qwen | 128K / 32K |
| Linux PC with no GPU, 32 GB | 25Q4_K_M (4-bit) by unsloth | 19Q4_K_M (4-bit) by Qwen | 4K / 4K |
| Windows PC with an NVIDIA GPU, 16 GB | 98Q4_K_M (4-bit) by unsloth | 248Q4_K_M (4-bit) by Qwen | 128K / 8K |
| Windows PC with an NVIDIA GPU, 8 GB | 15Q4_K_M (4-bit) by unsloth | 5.8Q4_K_M (4-bit) by Qwen | — / — |
| Windows PC with Intel graphics, 16 GB | 6.8Q4_K_M (4-bit) by unsloth | 32Q4_K_M (4-bit) by Qwen | — / 16K |
| Windows, 12 CPUs · 7 GB | 15Q4_K_M (4-bit) by unsloth | 5.8Q4_K_M (4-bit) by Qwen | 128K / 32K |