TokenMark: Local AI Config Tracker for Strix Halo & DGX Spark

Measured local-LLM tok/s on AMD Strix Halo, NVIDIA DGX Spark and Apple Mac, plus GPU cards in their own lane. Source-linked, updated daily.

Snapshot 2026-11-01 · 412 configs on AI boxes and 288 on GPU cards · 121 models. Every number is generated from the tracker snapshot and linked to where it was measured; nothing here is typed by hand.

AMD Strix Halo (Ryzen AI Max+ 395) · 183 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3 0.6B	Q8_0	llama.cpp/vulkan	266 tok/s (prefill 13112)	512	Structured table	source
Qwen3 0.6B	Q8_0	llama.cpp/rocm	208.7 tok/s (prefill 4666.1)	512	Structured table	source
LFM2.5-2.6B	Q4_K_M	llama.cpp/vulkan (speculative decode, ngram)	182.1 tok/s	8192	Extracted, source-linked	source
LFM2.5 8B-A1B	Q4_K_M	llama.cpp/vulkan	176.5 tok/s (prefill 3398.4)	512	Structured table	source
Qwen3.6-35B-A3B	UD-IQ4_XS	llama.cpp/vulkan (speculative decode, mtp-2)	127.1 tok/s		Extracted, source-linked	source

AMD Gorgon Halo (Ryzen AI Max+ PRO 495) · 0 single-stream configs

No single-stream configs recorded yet.

NVIDIA DGX Spark (GB10) · 129 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3 1.7B	Q4_K_M	llama.cpp	146.1 tok/s	2048	Extracted, source-linked	source
Qwen3.5-35B-A3B	NVFP4	atlas (speculative decode, mtp-2)	131 tok/s		Extracted, source-linked	source
Qwen3.6-35B-heretic	NVFP4	vllm	96 tok/s		Structured table	source
Qwen3.6-35B-A3B-NVFP4-Fast (Unsloth)	NVFP4	vllm	92.6 tok/s		Structured table	source
GPT-OSS-20B	Q4_K_XL	llama.cpp	90.5 tok/s		Structured table	source

Apple M-series Max (MacBook Pro / Mac Studio Max) · 32 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
llama-3.2-1b	MLX-4BIT	mlx	527.7 tok/s (prefill 17477)	512	Extracted, source-linked	source
llama-3.2-1b	Q4_K_M	llama.cpp	439.6 tok/s (prefill 16968)	512	Extracted, source-linked	source
Qwen3-4B	MLX-4BIT	mlx	186 tok/s (prefill 5506)	512	Extracted, source-linked	source
Qwen3-4B	Q4_K_M	llama.cpp	159.7 tok/s (prefill 4973)	512	Extracted, source-linked	source
Qwen3.6-35B-A3B	MLX-4BIT	mlx	151.1 tok/s (prefill 3333)	512	Extracted, source-linked	source

Apple Mac Studio (M-series Ultra) · 10 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.6-35B-A3B	MLX-4BIT	mlx	109.1 tok/s (prefill 6012)	32000	Extracted, source-linked	source
Qwen3.8-Flash-Next	MLX-4BIT	mlx (speculative decode, mtp)	76.4 tok/s		Extracted, source-linked	source
Qwen3.5-122B-A10B	MLX-4BIT	mlx	65.3 tok/s (prefill 2300)	32000	Extracted, source-linked	source
Qwen3-14B	MLX-4BIT	mlx	55.4 tok/s (prefill 2278)	32000	Extracted, source-linked	source
GLM-5.3-Flash		mlx (speculative decode, mtp)	55.2 tok/s		Extracted, source-linked	source

GPU cards, ranked on their own

GPU cards run the model from their own VRAM: very fast on what fits, but most cannot load what a 128 GB AI box can. Ranked on their own, never against the boxes. Every GPU-card config.

NVIDIA RTX 5090 (32GB) · 207 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.5-0.8B	Q4_0	llama.cpp/cuda	413 tok/s	8192	Structured table	source
Qwen3.5-0.8B	IQ4_NL	llama.cpp/cuda	410 tok/s	8192	Structured table	source
Qwen3.5-0.8B	UD-IQ3_XXS	llama.cpp/cuda	409.5 tok/s	8192	Structured table	source
Qwen3.5-0.8B	Q4_1	llama.cpp/cuda	407.9 tok/s	8192	Structured table	source
Qwen3.5-0.8B	IQ4_XS	llama.cpp/cuda	406.7 tok/s	8192	Structured table	source

NVIDIA H100 PCIe (80GB) · 19 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.6-35B-A3B	UD-Q4_K_M	llama.cpp/cuda	152.6 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q4_K_XL	llama.cpp/cuda	152.1 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q5_K_M	llama.cpp/cuda	151.3 tok/s	8192	Verified (ran it)	source
Qwen3.5-35B-A3B	Q4_K_M	llama.cpp/cuda	151.3 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q6_K	llama.cpp/cuda	146.1 tok/s	8192	Verified (ran it)	source

NVIDIA H100 SXM (80GB) · 15 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.6-35B-A3B	UD-Q4_K_M	llama.cpp/cuda	188.1 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q5_K_M	llama.cpp/cuda	185.3 tok/s	8192	Verified (ran it)	source
Qwen3.5-35B-A3B	Q4_K_M	llama.cpp/cuda	185.2 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q4_K_XL	llama.cpp/cuda	183.1 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q6_K	llama.cpp/cuda	178.2 tok/s	8192	Verified (ran it)	source

NVIDIA A100 40GB · 19 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.6-35B-A3B	UD-Q4_K_M	llama.cpp/cuda	124 tok/s	8192	Verified (ran it)	source
Qwen3.5-35B-A3B	Q4_K_M	llama.cpp/cuda	122.8 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q4_K_XL	llama.cpp/cuda	122.1 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q5_K_M	llama.cpp/cuda	120.9 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q6_K	llama.cpp/cuda	118.6 tok/s	8192	Verified (ran it)	source

NVIDIA A10 (24GB) · 8 single-stream configs

Model	Quant	Backend	Decode	Context	Trust	Source
Qwen3.5-35B-A3B	Q4_K_M	llama.cpp/cuda	97.9 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q4_K_M	llama.cpp/cuda	95.4 tok/s	8192	Verified (ran it)	source
Qwen3.6-35B-A3B	UD-Q4_K_XL	llama.cpp/cuda	91.3 tok/s	8192	Verified (ran it)	source
Qwen3.8-27B	INT4	vllm	28.4 tok/s (prefill 1111.3)	32768	Verified (ran it)	source
Qwen3.8-27B	UD-Q4_K_M	llama.cpp/cuda	24.9 tok/s	8192	Verified (ran it)	source

Also on TokenMark: daily local-AI releases, explained · the people who measured these numbers · guides · real on-prem deployments.

More: all configs · hardware · news · contributors · methodology · llms.txt · llms-full.txt · API (OpenAPI) · JSON snapshot. Built by Altronis, private on-prem AI, Singapore.