Releases·Tool Release
Qwen3.8 27B on one AMD R9700: 569,878-token reusable cache, 262K context, ~180 tok/s via W3A4 3-bit weights
Eliovp BV published a new coding mode for its Paiton vLLM plugin that runs Qwen3.8 27B on a single AMD Radeon AI PRO R9700 (32 GB) with 262,144-token context per request and 569,878 tokens of reusable prefix cache. Coding-agent turns start up to 12× sooner and full sessions finish ~6× faster because only new tokens are re-prefilled. The release pairs a selective 3-bit weight quantization scheme with unchanged MXFP4 as the more accurate option.
11 engagement1 sourceSun, Oct 4, 2026, 01:12 PM
AI Overview
- Eliovp's Paiton vLLM plugin runs Qwen3.8 27B on a single Radeon AI PRO R9700 (32 GB), caching 569,878 reusable tokens at 262,144-token context per request
- 3-bit weights quantize only the 24.3B MLP, attention, and Gated DeltaNet projection matrices (rotated, GPTQ-calibrated, ~3.1 bits/weight), with FP8 decode activations; MXFP4 remains available
- Coding-agent sessions finish about 6.1× faster and later turns start up to 12× sooner, with 91–92% of prompt tokens served from the prefix cache
- Paired accuracy: 3-bit trails MXFP4 by 2.9 points on MMLU-Pro (95% CI −4.8 to −0.9); GSM8K (95.45) and HumanEval (92.07) stay within noise
- An opt-in --mode long-512k serves 524,288-token requests, retrieving 4/4 planted facts at 500K, with a 594,290-token cache
- Only the large projection matrices are 3-bit; embeddings, output head, and norms come from the MXFP4 release
Related
Releases·Tool Release
LlamAmpere v0.4 hits 95+ TPS through 100K generated for Qwen3.8 27B at 262K ctx on a single 3090
Developer JakeATX released LlamAmpere v0.4, a Llama.cpp fork with Ampere-specific optimizations, achieving 95+ tokens/second over 100K generated tokens for a Qwen3.8 27B model at 262K context on one RTX 3090. The update delivers ~10% speed gains and 10%+ more max context versus the prior version, with vLLM as the closest competing runtime. The author also quantifies the quality cost of the TQ5/TQ4 KV cache quantization used.
Developer JakeATX released LlamAmpere v0.4, a Llama.cpp fork with Ampere-specific optimizations, achieving 95+ tokens/second over 100K generated tokens for a Qwen3.8 27B model at 262K context on one RTX 3090. The update delivers ~10% speed gains and 10%+ more max context versus the prior version, with vLLM as the closest competing runtime. The author also quantifies the quality cost of the TQ5/TQ4 KV cache quantization used.
Updated Mon, Sep 28, 2026, 06:35 PM — v0.4 pushed with ~10% speed gain and 262K context; KVaRN planned for v0.5 along with 12GB card focus
57 engagement1 sourceredditMon, Sep 28, 2026, 06:35 PM
Releases·Tool Release
akopytko releases Blackwell-native NVFP4 quant of Qwen3.8 27B, 50% faster than Q4
A new GGUF quant of Qwen3.8 27B uses Blackwell-native NVFP4 with a prefill-optimized design, achieving 50% faster prefill than a Q4 quant of the same memory footprint and 4-7% faster than other NVFP4 quants on an RTX 5090. It includes a quantized MTP draft head and is available on Hugging Face.
A new GGUF quant of Qwen3.8 27B uses Blackwell-native NVFP4 with a prefill-optimized design, achieving 50% faster prefill than a Q4 quant of the same memory footprint and 4-7% faster than other NVFP4 quants on an RTX 5090. It includes a quantized MTP draft head and is available on Hugging Face.
166 engagement1 sourceredditFri, Aug 21, 2026, 09:19 AM
Releases·Tool Release
Dev releases MegaCapybara, an RTX5090 inference engine for Qwen3.8 27B claiming 2x Ninfer decode speed
A developer announced MegaCapybara, a purpose-built inference engine for the RTX 5090 focused on Qwen3.8 27B, with weights on Hugging Face and code on GitHub. It claims roughly twice the decode speed of ninfer, the prior SOTA engine for that GPU, and adds agent-serving, cache management, and benchmark-verified weight metadata. Source code and a weights builder will follow later.
A developer announced MegaCapybara, a purpose-built inference engine for the RTX 5090 focused on Qwen3.8 27B, with weights on Hugging Face and code on GitHub. It claims roughly twice the decode speed of ninfer, the prior SOTA engine for that GPU, and adds agent-serving, cache management, and benchmark-verified weight metadata. Source code and a weights builder will follow later.
62 engagement1 sourceredditSat, Oct 3, 2026, 01:33 AM
Social·Community
Community benchmarks Qwen 3.8 27B quants across MLX, GGUF, and NVFP4
Multiple community teams released and benchmarked quantized versions of Qwen 3.8 27B, including Atomic Dynamic GGUF quants on RTX PRO 6000, MLX vision quants, and a 5.5-bit PrismaAqua quant. Results show expected trade-offs between size, accuracy, and speed, with the model performing well on 3D scene tasks and professional workloads.
Multiple community teams released and benchmarked quantized versions of Qwen 3.8 27B, including Atomic Dynamic GGUF quants on RTX PRO 6000, MLX vision quants, and a 5.5-bit PrismaAqua quant. Results show expected trade-offs between size, accuracy, and speed, with the model performing well on 3D scene tasks and professional workloads.
Updated Wed, Aug 26, 2026, 07:32 PM — Newest post (Aug 26) details MLX vision quants with distillation-based low-bit builds, comparing against other community
120470 engagement5 sourcesredditTue, Aug 18, 2026, 01:53 AM
Releases·Tool Release
Developer's INT8 vLLM fork hits 972 tok/s TG on Qwen3.8 27B with 4x MI100
A developer released a complete INT8 serving stack for Qwen3.8 27B based on vLLM, AITER, and a GPTQ INT8 quant with DFlash2. On a 4x MI100 rig costing $6.5k, it achieves 972 tok/s time-to-first-token and 5,680 tokens/s per prompt, up from 15 tok/s on stock vLLM. This enables older INT8-centric hardware to run modern models efficiently.
A developer released a complete INT8 serving stack for Qwen3.8 27B based on vLLM, AITER, and a GPTQ INT8 quant with DFlash2. On a 4x MI100 rig costing $6.5k, it achieves 972 tok/s time-to-first-token and 5,680 tokens/s per prompt, up from 15 tok/s on stock vLLM. This enables older INT8-centric hardware to run modern models efficiently.
57 engagement1 sourceredditWed, Aug 26, 2026, 08:54 PM