COMPARE
All runs, side by side
6 runs. Click a column to sort; the best warm decode is highlighted.
DECODE TRENDS · WARM T/S
Qwen3.8-27B 56.1
click a column to sort
| Date | Model · quant | Engine | Warm decode (t/s) | TTFT warm (s) | VRAM peak (GB) | Verdict / note |
|---|---|---|---|---|---|---|
| 2026-10-16 | Qwen3.6-35B-A3B · GPTQ-Int4 | vLLM XPU | 105.8 | 1.50 | 31.4 | first baseline. MTP×4 (accept 51.8%) |
| 2026-10-16 | Qwen3.8-27B · EXL3 4.00bpw | exl3xpu | 57.0 | 1.43 | 30.7 | second engine; decode gap vs baseline confounded (model + quant + MTP depth differ) |
| 2026-10-17 | Qwen3.8-27B · EXL3 4.00bpw | exl3xpu | 56.6 | 1.40 | 31.4 | revert · A/B batched-tokens 4096→8192: no speedup, prefill unchanged, KV pool −28k tok |
| 2026-10-17 | Qwen3.8-27B · GGUF IQ3_S (GSQ) | llama.cpp SYCL/XPU | 44.9 | 23.45 | 20.9 | EXL3 faster on decode (57.0 vs 44.9); prefill 6.3× slower than other engines |
| 2026-10-17 | Qwen3.8-27B · GGUF IQ3_S (GSQ) | llama.cpp SYCL/XPU | 56.8 | 16.00 | 29.8 | ubatch 4096: no prefill gain; --no-kv-offload = crash culprit |
| 2026-10-17 | Qwen3.8-27B · GGUF IQ3_S (GSQ) | llama.cpp SYCL/XPU | 56.1 | 16.65 | 23.4 | keep · KV q8_0: stable, no prefill cost, spec accept up (83% warm); only KV dtype fitting ctx 262k |
KEY FINDINGS
Qwen3.6-35B-A3B · vLLM XPU
105.8 t/s with MTP speculation (raw non-spec est. 40–60). First baseline. MoE A3B active-parameter profile makes comparison against dense models confounded.
Qwen3.8-27B · exl3xpu
No measurable speedup from raising max_num_batched_tokens from 4096 to 8192; KV pool shrank 28k tokens. Recommend reverting to 4096. verdict: revert
Qwen3.8-27B · llama.cpp SYCL/XPU
Stable run, only KV dtype fitting ctx 262k (~27 GB). q8_0: no prefill cost, spec accept up (warm 83%). verdict: keep