Byram AI Lab

Byram AI Labhttps://byram.ai/2026-10-17T00:00:00ZQwen3.6-35B-A3B · GPTQ-Int4 · vLLM XPUhttps://byram.ai/2026-10-16-qwen36-35b-a3b-vllm-xpu/2026-10-16T00:00:00Z

Qwen3.6-35B-A3B with GPTQ-Int4 on vLLM XPU, 105.8 t/s warm decode with MTP speculation.

Qwen3.8-27B · EXL3 4.00bpw · exl3xpuhttps://byram.ai/2026-10-16-qwen38-27b-exl3xpu/2026-10-16T00:00:00Z

Qwen3.8-27B EXL3 4.00bpw on exl3xpu, 57.0 t/s warm decode, 1.43s TTFT.

Qwen3.8-27B · EXL3 · batch 4096→8192https://byram.ai/2026-10-17-qwen38-27b-exl3xpu-batched-tokens-8192/2026-10-17T00:00:00Z

A/B: max_num_batched_tokens 4096→8192. No speedup; KV pool shrank 28k tokens. Recommend revert.

Qwen3.8-27B · GGUF IQ3_S · llama.cpphttps://byram.ai/2026-10-17-qwen38-27b-gsq-iq3s-llamacpp/2026-10-17T00:00:00Z

Qwen3.8-27B GSQ IQ3_S GGUF on llama.cpp, 44.9 t/s warm, same model as EXL3 runs but 6.3× slower prefill.

Qwen3.8-27B · GGUF · ubatch 4096 + KV q8_0https://byram.ai/2026-10-17-qwen38-27b-gsq-iq3s-llamacpp-batch4k-kvq8/2026-10-17T00:00:00Z

Qwen3.8-27B GGUF on llama.cpp with ubatch 4096 + KV q8_0, 56.8 t/s warm, multi-variable run.

Qwen3.8-27B · GGUF · KV q8_0https://byram.ai/2026-10-17-qwen38-27b-gsq-iq3s-llamacpp-kvq8/2026-10-17T00:00:00Z

Qwen3.8-27B GGUF on llama.cpp with KV q8_0, 56.1 t/s warm, stable run with high spec acceptance.