Hardware › RTX 5090 local LLM benchmarks: 207 measured configs

RTX 5090 local LLM benchmarks: 207 measured configs

207 measured local-LLM configs on NVIDIA RTX 5090 (32GB); fastest 14B+ model GPT-OSS-20B at 355.6 tok/s. Official specs, every number linked to its source.

Snapshot 2026-10-15 · 394 configs on AI boxes and 423 on GPU cards · 123 models. Every number is generated from the tracker snapshot and linked to where it was measured; nothing here is typed by hand.

Quick answers

What is the fastest model on RTX 5090?: GPT-OSS-20B at 355.6 tok/s single-stream (Q3_K_S, llama.cpp/cuda), source last checked 2026-10-15. Models under 14B parameters are left out of this answer. source
What is the largest model measured on RTX 5090?: Qwen3.6-35B-A3B (35B parameters) at 210.9 tok/s single-stream (UD-IQ4_NL, llama.cpp/cuda), source last checked 2026-10-15. source
How many configs does TokenMark track for RTX 5090?: 207 single-stream configs across 13 models, each linked to the README or model card it came from.

Measured single-stream configs on NVIDIA RTX 5090 (32GB)

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.5-0.8B	Q4_0	llama.cpp/cuda	413 tok/s	8192	Structured table	source
Qwen3.5-0.8B	IQ4_NL	llama.cpp/cuda	410 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-IQ3_XXS	llama.cpp/cuda	409.5 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q4_1	llama.cpp/cuda	407.9 tok/s	8192	Structured table	source
Qwen3.5-0.8B	IQ4_XS	llama.cpp/cuda	406.7 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-Q4_K_XL	llama.cpp/cuda	405.5 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q5_K_S	llama.cpp/cuda	402.7 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q4_K_S	llama.cpp/cuda	402.2 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q4_K_M	llama.cpp/cuda	401.2 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-Q5_K_XL	llama.cpp/cuda	399.1 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q5_K_M	llama.cpp/cuda	397.8 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-Q6_K_XL	llama.cpp/cuda	397 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q8_0	llama.cpp/cuda	396.6 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-Q3_K_XL	llama.cpp/cuda	394.9 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q6_K	llama.cpp/cuda	392.8 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q3_K_M	llama.cpp/cuda	388.6 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q3_K_S	llama.cpp/cuda	381.6 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-Q8_K_XL	llama.cpp/cuda	370.8 tok/s	8192	Structured table	source
Qwen3.5-2B	UD-IQ3_XXS	llama.cpp/cuda	366.3 tok/s	8192	Structured table	source
gpt-oss-20b	Q3_K_S	llama.cpp/cuda	355.6 tok/s	8192	Structured table	source
Qwen3.5-2B	IQ4_XS	llama.cpp/cuda	352.4 tok/s	8192	Structured table	source
Qwen3.5-2B	Q4_0	llama.cpp/cuda	352.2 tok/s	8192	Structured table	source
Qwen3.5-2B	IQ4_NL	llama.cpp/cuda	350.9 tok/s	8192	Structured table	source
Qwen3.5-2B	UD-Q3_K_XL	llama.cpp/cuda	349.8 tok/s	8192	Structured table	source
Qwen3.5-0.8B	BF16	llama.cpp/cuda	349.6 tok/s	8192	Structured table	source

More: all configs · hardware · news · contributors · methodology · llms.txt · llms-full.txt · API (OpenAPI) · JSON snapshot. Built by Altronis, private on-prem AI, Singapore.