Configs: Strix Halo & DGX Spark LLM Benchmark Leaderboard

Searchable, source-linked local-LLM configs on Strix Halo, DGX Spark and Mac, plus a separate GPU-card lane. Filterable by trust.

Snapshot 2026-10-15 · 394 configs on AI boxes and 423 on GPU cards · 123 models. Every number is generated from the tracker snapshot and linked to where it was measured; nothing here is typed by hand.

AMD Strix Halo (Ryzen AI Max+ 395) · 178 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3 0.6B	Q8_0	llama.cpp/vulkan	266 tok/s (prefill 13112)	512	Structured table	source
Qwen3 0.6B	Q8_0	llama.cpp/rocm	208.7 tok/s (prefill 4666.1)	512	Structured table	source
LFM2.5-2.6B	Q4_K_M	llama.cpp/vulkan (speculative decode, ngram)	182.1 tok/s	8192	Extracted, source-linked	source
LFM2.5 8B-A1B	Q4_K_M	llama.cpp/vulkan	176.5 tok/s (prefill 3398.4)	512	Structured table	source
Ornith-1.5-35B-A3B	FP4	speculative decode, mtp	105.6 tok/s		Owner-submitted	source
Qwen3-30B-A3B-Instruct-2507	IQ4_XS	llama.cpp/vulkan	103.2 tok/s (prefill 1438.1)	512	Structured table	source
Qwen3-Coder 30B-A3B	Q4_K_S	llama.cpp/vulkan	98 tok/s (prefill 1406.5)	512	Structured table	source
Qwen3-Coder 30B-A3B	UD-Q4_K_XL	llama.cpp/vulkan	97.1 tok/s (prefill 1400)	512	Structured table	source
NVIDIA Nemotron 3.5 Lightning 30B	FP4	speculative decode, mtp	95.2 tok/s		Owner-submitted	source
Qwen3-Coder 30B-A3B	IQ4_XS	llama.cpp/vulkan	90.4 tok/s (prefill 1372.3)	512	Structured table	source
Qwen3 30B-A3B NEO-MAX	IQ4_XS	llama.cpp/vulkan	87.4 tok/s (prefill 1396.1)	512	Structured table	source
Qwen3.6-35B-A3B	Q4_0	llama.cpp/vulkan	81.3 tok/s (prefill 1243.5)	512	Structured table	source
llm-jp-4.1-32b-a3b-thinking	Q4_K_M	llama.cpp/vulkan (depth-0)	81 tok/s (prefill 1980.2)		Verified (ran it)	source
Qwen3-30B-A3B	UD-Q4_K_XL	llama.cpp/rocm	79.1 tok/s (prefill 687.9)		Structured table	source
Nemotron Cascade 2 30B-A3B	IQ4_XS	llama.cpp/vulkan	79 tok/s (prefill 1325.3)	512	Structured table	source

AMD Gorgon Halo (Ryzen AI Max+ PRO 495) · 0 single-stream configs

No single-stream configs recorded yet.

NVIDIA DGX Spark (GB10) · 128 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3 1.7B	Q4_K_M	llama.cpp	146.1 tok/s	2048	Extracted, source-linked	source
Qwen3.5-35B-A3B	NVFP4	atlas (speculative decode, mtp-2)	131 tok/s		Extracted, source-linked	source
Qwen3.6-35B-heretic	NVFP4	vllm	96 tok/s		Structured table	source
Qwen3.6-35B-A3B-NVFP4-Fast (Unsloth)	NVFP4	vllm	92.6 tok/s		Structured table	source
GPT-OSS-20B	Q4_K_XL	llama.cpp	90.5 tok/s		Structured table	source
Ministral 3B	Q4_K_M	llama.cpp	86.6 tok/s	2048	Extracted, source-linked	source
Qwen3.6-35B-A3B	FP8	metrale (speculative decode, mtp-1)	85 tok/s	2048	Extracted, source-linked	source
Ornith-1.0-35B-AEON	NVFP4	vllm	84.8 tok/s		Structured table	source
Qwen3 30B (MoE)	Q4_K_M	llama.cpp	83.8 tok/s	2048	Extracted, source-linked	source
Qwen3.6-35B-A3B	NVFP4	vllm	77.4 tok/s		Structured table	source
Qwen3.6-35B-A3B	UD-IQ3_XXS	llama.cpp/cuda	66.3 tok/s	8192	Structured table	source
Qwen3.6-35B-A3B	UD-IQ3_S	llama.cpp/cuda	65.6 tok/s	8192	Structured table	source
Qwen3.6-35B-A3B	UD-Q3_K_XL	llama.cpp/cuda	64.1 tok/s	8192	Structured table	source
Qwen3.6-35B-A3B	UD-Q3_K_M	llama.cpp/cuda	64 tok/s	8192	Structured table	source
Qwen3.6-35B-A3B	UD-IQ4_NL	llama.cpp/cuda	63.8 tok/s	8192	Structured table	source

Apple M-series Max (MacBook Pro / Mac Studio Max) · 29 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
llama-3.2-1b	MLX-4BIT	mlx	527.7 tok/s (prefill 17477)	512	Extracted, source-linked	source
llama-3.2-1b	Q4_K_M	llama.cpp	439.6 tok/s (prefill 16968)	512	Extracted, source-linked	source
Qwen3-4B	MLX-4BIT	mlx	186 tok/s (prefill 5506)	512	Extracted, source-linked	source
Qwen3-4B	Q4_K_M	llama.cpp	159.7 tok/s (prefill 4973)	512	Extracted, source-linked	source
Qwen3.6-35B-A3B	MLX-4BIT	mlx	151.1 tok/s (prefill 3333)	512	Extracted, source-linked	source
Qwen3-30B-A3B	Q4_K_M	llama.cpp	148.5 tok/s (prefill 3497)	512	Extracted, source-linked	source
Qwen3-30B-A3B	MLX-4BIT	mlx	147.2 tok/s (prefill 3458)	512	Extracted, source-linked	source
Qwen3.6-35B-A3B	Q4_K_M	llama.cpp	124.3 tok/s (prefill 3343)	512	Extracted, source-linked	source
Qwen3-8B	MLX-4BIT	mlx	110.5 tok/s (prefill 3151)	512	Extracted, source-linked	source
Qwen3-8B	Q4_K_M	llama.cpp	97.7 tok/s (prefill 2745)	512	Extracted, source-linked	source
gemma-3-12b	MLX-4BIT	mlx	66.9 tok/s (prefill 1835)	512	Extracted, source-linked	source
Qwen3-14B	MLX-4BIT	mlx	62.5 tok/s (prefill 1739)	512	Extracted, source-linked	source
gemma-3-12b	Q4_K_M	llama.cpp	61.5 tok/s (prefill 1721)	512	Extracted, source-linked	source
Qwen3.8-Flash-Next	Q4-INF	mlx	57.5 tok/s		Extracted, source-linked	source
Qwen3-14B	Q4_K_M	llama.cpp	57.4 tok/s (prefill 1584)	512	Extracted, source-linked	source

Apple Mac Studio (M-series Ultra) · 10 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.6-35B-A3B	MLX-4BIT	mlx	109.1 tok/s (prefill 6012)	32000	Extracted, source-linked	source
Qwen3.8-Flash-Next	MLX-4BIT	mlx (speculative decode, mtp)	76.4 tok/s		Extracted, source-linked	source
Qwen3.5-122B-A10B	MLX-4BIT	mlx	65.3 tok/s (prefill 2300)	32000	Extracted, source-linked	source
Qwen3-14B	MLX-4BIT	mlx	55.4 tok/s (prefill 2278)	32000	Extracted, source-linked	source
GLM-5.3-Flash		mlx (speculative decode, mtp)	55.2 tok/s		Extracted, source-linked	source
Qwen3.8-27B	MLX-4BIT	mlx	42.6 tok/s (prefill 1526)	32000	Extracted, source-linked	source
GLM-5.3-Flash	MLX-4BIT	mlx	34.2 tok/s		Extracted, source-linked	source
DeepSeek-V4-Flash-0731-MXFP4-MLX-Abliterated	MXFP4	mlx	29.6 tok/s (prefill 454)	1048576	Extracted, source-linked	source
GLM-5.3-Flash Abliterated	MLX-4BIT	mlx (speculative decode, mtp)	24 tok/s (prefill 365)	16384	Extracted, source-linked	source
Gemma-4-31B	BF16	mlx	13.6 tok/s (prefill 1217)	32000	Extracted, source-linked	source

GPU cards, ranked on their own

GPU cards run the model from their own VRAM: very fast on what fits, but most cannot load what a 128 GB AI box can. Ranked on their own, never against the boxes. Every GPU-card config.

NVIDIA RTX 5090 (32GB) · 207 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.5-0.8B	Q4_0	llama.cpp/cuda	413 tok/s	8192	Structured table	source
Qwen3.5-0.8B	IQ4_NL	llama.cpp/cuda	410 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-IQ3_XXS	llama.cpp/cuda	409.5 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q4_1	llama.cpp/cuda	407.9 tok/s	8192	Structured table	source
Qwen3.5-0.8B	IQ4_XS	llama.cpp/cuda	406.7 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-Q4_K_XL	llama.cpp/cuda	405.5 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q5_K_S	llama.cpp/cuda	402.7 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q4_K_S	llama.cpp/cuda	402.2 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q4_K_M	llama.cpp/cuda	401.2 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-Q5_K_XL	llama.cpp/cuda	399.1 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q5_K_M	llama.cpp/cuda	397.8 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-Q6_K_XL	llama.cpp/cuda	397 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q8_0	llama.cpp/cuda	396.6 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-Q3_K_XL	llama.cpp/cuda	394.9 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q6_K	llama.cpp/cuda	392.8 tok/s	8192	Structured table	source

NVIDIA H100 PCIe (80GB) · 19 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.6-35B-A3B	UD-Q4_K_M	llama.cpp/cuda	152.6 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q4_K_XL	llama.cpp/cuda	152.1 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q5_K_M	llama.cpp/cuda	151.3 tok/s	8192	Verified (ran it)	source
Qwen3.5-35B-A3B	Q4_K_M	llama.cpp/cuda	151.3 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q6_K	llama.cpp/cuda	146.1 tok/s	8192	Verified (ran it)	source
gemma-4-26B-A4B-it	UD-Q8_K_XL	llama.cpp/cuda	138 tok/s	8192	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm (speculative decode, mtp-4)	103.7 tok/s	65536	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm (speculative decode, mtp-2)	101.4 tok/s	65536	Verified (ran it)	source
Qwen3.8-27B	FP8	vllm (speculative decode, mtp-4)	95.9 tok/s	65536	Verified (ran it)	source
Qwen3.8-27B	FP8	vllm (speculative decode, mtp-2)	91.1 tok/s	65536	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm	60 tok/s (prefill 4645.6)	65536	Verified (ran it)	source
Qwen3.8-27B	UD-Q4_K_M	llama.cpp/cuda	56.7 tok/s	8192	Verified (ran it)	source
Qwen3.8-27B	UD-Q4_K_XL	llama.cpp/cuda	54.4 tok/s	8192	Verified (ran it)	source
Qwen3.6-27B	Q4_K_M	llama.cpp/cuda	52.7 tok/s	8192	Verified (ran it)	source
Qwen3.5-27B	Q4_K_M	llama.cpp/cuda	52.3 tok/s	8192	Verified (ran it)	source

NVIDIA H100 SXM (80GB) · 15 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.6-35B-A3B	UD-Q4_K_M	llama.cpp/cuda	188.1 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q5_K_M	llama.cpp/cuda	185.3 tok/s	8192	Verified (ran it)	source
Qwen3.5-35B-A3B	Q4_K_M	llama.cpp/cuda	185.2 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q4_K_XL	llama.cpp/cuda	183.1 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q6_K	llama.cpp/cuda	178.2 tok/s	8192	Verified (ran it)	source
gemma-4-26B-A4B-it	UD-Q8_K_XL	llama.cpp/cuda	171.7 tok/s	8192	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm (speculative decode, mtp-4)	161.8 tok/s	65536	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm (speculative decode, mtp-2)	145.8 tok/s	65536	Verified (ran it)	source
Qwen3.8-27B	FP8	vllm (speculative decode, mtp-4)	141.1 tok/s	65536	Verified (ran it)	source
Qwen3.8-27B	FP8	vllm (speculative decode, mtp-2)	139.1 tok/s	65536	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm	85.3 tok/s (prefill 11225.3)	65536	Verified (ran it)	source
Qwen3.8-27B	FP8	vllm	74.3 tok/s (prefill 12149)	65536	Verified (ran it)	source
Qwen3.5-27B	Q4_K_M	llama.cpp/cuda	68.5 tok/s	8192	Verified (ran it)	source
Qwen3.6-27B	Q4_K_M	llama.cpp/cuda	68.1 tok/s	8192	Verified (ran it)	source
Qwen3.6-27B	Q8_0	llama.cpp/cuda	60.2 tok/s	8192	Verified (ran it)	source

NVIDIA A100 40GB · 19 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.6-35B-A3B	UD-Q4_K_M	llama.cpp/cuda	124 tok/s	8192	Verified (ran it)	source
Qwen3.5-35B-A3B	Q4_K_M	llama.cpp/cuda	122.8 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q4_K_XL	llama.cpp/cuda	122.1 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q5_K_M	llama.cpp/cuda	120.9 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q6_K	llama.cpp/cuda	118.6 tok/s	8192	Verified (ran it)	source
gemma-4-26B-A4B-it	UD-Q8_K_XL	llama.cpp/cuda	99.9 tok/s	8192	Verified (ran it)	source
Qwen3.8-27B	FP8	vllm (speculative decode, mtp-4)	81.2 tok/s	32768	Verified (ran it)	source
Qwen3.8-27B	FP8	vllm (speculative decode, mtp-2)	76.5 tok/s	32768	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm (speculative decode, mtp-4)	72.2 tok/s	65536	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm (speculative decode, mtp-2)	69 tok/s	65536	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm	62.3 tok/s (prefill 3142.5)	65536	Verified (ran it)	source
Qwen3.8-27B	UD-Q4_K_M	llama.cpp/cuda	45.5 tok/s	8192	Verified (ran it)	source
Qwen3.8-27B	UD-Q4_K_XL	llama.cpp/cuda	43.5 tok/s	8192	Verified (ran it)	source
Qwen3.6-27B	Q4_K_M	llama.cpp/cuda	42.4 tok/s	8192	Verified (ran it)	source
Qwen3.5-27B	Q4_K_M	llama.cpp/cuda	42.4 tok/s	8192	Verified (ran it)	source

NVIDIA A10 (24GB) · 8 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.5-35B-A3B	Q4_K_M	llama.cpp/cuda	97.9 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q4_K_M	llama.cpp/cuda	95.4 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q4_K_XL	llama.cpp/cuda	91.3 tok/s	8192	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm	28.4 tok/s (prefill 1111.3)	32768	Verified (ran it)	source
Qwen3.8-27B	UD-Q4_K_M	llama.cpp/cuda	24.9 tok/s	8192	Verified (ran it)	source
Qwen3.6-27B	Q4_K_M	llama.cpp/cuda	24.1 tok/s	8192	Verified (ran it)	source
Qwen3.5-27B	Q4_K_M	llama.cpp/cuda	23.5 tok/s	8192	Verified (ran it)	source
Qwen3.8-27B	UD-Q4_K_XL	llama.cpp/cuda	23.5 tok/s	8192	Verified (ran it)	source

More: all configs · hardware · news · contributors · methodology · llms.txt · llms-full.txt · API (OpenAPI) · JSON snapshot. Built by Altronis, private on-prem AI, Singapore.