Inference · hardware

What runs on Windows PC with an NVIDIA GPU, 16 GB.

Every open model a machine of this kind has run on the network, at its fastest build, fastest first. Speed is tokens a second, measured and signed by the machine itself.

2models measured
248fastest tok/s
2machines
ModelFastest build heretok/sContext$ / M tokens
Qwen2.5 0.5B InstructQ4_K_M (4-bit) by Qwen24832K$1.04
Gemma 4 E2BQ4_K_M (4-bit) by unsloth98128K$2.65