Releases·Tool Release
ATX-Swift-1.5-Qwen3.8-27B-Uncensored-MTP i1-Q5_K_M GGUF sustains 50-65+ t/s at full 131k context on one RTX 3090
A developer released ATX-Swift-1.5-Qwen3.8-27B-Uncensored-MTP, an asymmetrically quantized GGUF suite that keeps a 27B model at i1-Q5_K_M precision with a full 131,072-token context entirely in 24GB VRAM. Isolating the Multi-Token Prediction draft head at Q8_0 and compressing the KV cache to turbo4/5 raises deep-context throughput to 50-53.7 t/s sustained (63 t/s peaks) versus ~45 t/s for standard quants on an RTX 3090.
5 engagement1 sourceWed, Sep 30, 2026, 04:42 PM
AI Overview
- ATX-Swift-1.5 (i1-Q5_K_M GGUF) runs a 27B model with full 131,072-token context entirely in 24GB VRAM: 19.2GB weights, ~3.8GB KV cache, 23.4GB total.
- Asymmetric quant isolates the MTP draft head (blk.64) at Q8_0 and attention layers at Q6_K/Q8_0, raising draft acceptance from ~64% to 75-82%.
- Turbo KV cache compression (-ctk turbo5 -ctv turbo4) shrinks the 131k-token KV cache to ~3.5-4.2GB, avoiding Q3/Q2 compromises or CPU offload.
- Benchmarks on RTX 3090 at ~80k-98k context show 50-54 t/s sustained vs 45 t/s for standard mradermacher quants, with burst peaks of 58-63 t/s.
- An MTP vocabulary shortlist (atx_65536.txt) restricting draft projections to 65,536 tokens avoids a 73% compute penalty on Ampere Tensor Cores.
- Quant suite spans i1-Q8_0, i1-Q6_K, i1-Q5_K_M, i1-Q4_K_M plus mmproj-BF16.gguf for multimodal vision; hosted on Hugging Face.
Related
Social·Community
48Gb VRAM speed AND quality ! (Qwen 3.8 27B Swift 1.5 W8A16)
A Reddit user published 'Swift 1.5', a custom combined AWQ+GPTQ 8-bit quant of Qwen 3.8 27B on Hugging Face, aimed at users with 48Gb of VRAM. They report nearly 100 tok/s decode with MTP on 2 RTX 3090 using vLLM. The quant is a practical option for practitioners needing both speed and quality on modest multi-GPU hardware.
A Reddit user published 'Swift 1.5', a custom combined AWQ+GPTQ 8-bit quant of Qwen 3.8 27B on Hugging Face, aimed at users with 48Gb of VRAM. They report nearly 100 tok/s decode with MTP on 2 RTX 3090 using vLLM. The quant is a practical option for practitioners needing both speed and quality on modest multi-GPU hardware.
20 engagement1 sourceredditSat, Oct 10, 2026, 04:11 PM
Releases·Tool Release
LlamAmpere v0.4 hits 95+ TPS through 100K generated for Qwen3.8 27B at 262K ctx on a single 3090
Developer JakeATX released LlamAmpere v0.4, a Llama.cpp fork with Ampere-specific optimizations, achieving 95+ tokens/second over 100K generated tokens for a Qwen3.8 27B model at 262K context on one RTX 3090. The update delivers ~10% speed gains and 10%+ more max context versus the prior version, with vLLM as the closest competing runtime. The author also quantifies the quality cost of the TQ5/TQ4 KV cache quantization used.
Developer JakeATX released LlamAmpere v0.4, a Llama.cpp fork with Ampere-specific optimizations, achieving 95+ tokens/second over 100K generated tokens for a Qwen3.8 27B model at 262K context on one RTX 3090. The update delivers ~10% speed gains and 10%+ more max context versus the prior version, with vLLM as the closest competing runtime. The author also quantifies the quality cost of the TQ5/TQ4 KV cache quantization used.
Updated Mon, Sep 28, 2026, 06:35 PM — v0.4 pushed with ~10% speed gain and 262K context; KVaRN planned for v0.5 along with 12GB card focus
57 engagement1 sourceredditMon, Sep 28, 2026, 06:35 PM
Products·launched
GPU memory checker that estimates VRAM needs including context length and concurrent chats
ComputeAtlas.ai is a tool for planning local AI hardware. It breaks GPU memory use into model file, KV cache, and runtime allowance based on model, GGUF quant, context length, and number of concurrent chats, showing whether a config fits a given GPU. Built with AI-assisted coding.
ComputeAtlas.ai is a tool for planning local AI hardware. It breaks GPU memory use into model file, KV cache, and runtime allowance based on model, GGUF quant, context length, and number of concurrent chats, showing whether a config fits a given GPU. Built with AI-assisted coding.
4 engagement1 sourceredditFri, Oct 9, 2026, 02:30 PM
Social·Community
SWE builds 3x RTX 3090, 64GB VRAM local AI coding PC running Qwen3.8-27B via llama.cpp
A software engineer detailed a DIY workstation with three RTX 3090s (~62GB usable VRAM) built to replace paid OpenAI/Anthropic subscriptions for coding agents. The build runs a Qwen3.8-27B Q8_0 GGUF quant through a self-compiled llama.cpp server with 262k context, at 44-46 t/s dropping to 28-30 t/s as context fills. It shows local multi-GPU setups are now practical for real software engineering workflows, though with nontrivial hardware constraints around PSUs, risers, and cooling.
A software engineer detailed a DIY workstation with three RTX 3090s (~62GB usable VRAM) built to replace paid OpenAI/Anthropic subscriptions for coding agents. The build runs a Qwen3.8-27B Q8_0 GGUF quant through a self-compiled llama.cpp server with 262k context, at 44-46 t/s dropping to 28-30 t/s as context fills. It shows local multi-GPU setups are now practical for real software engineering workflows, though with nontrivial hardware constraints around PSUs, risers, and cooling.
70 engagement1 sourceredditSun, Sep 13, 2026, 02:16 PM
Social·Community
MTP slows multi-GPU decode: Qwen3.6 35B A3B on 4x RTX3090 hits 82t/s without MTP vs 70t/s with
A Reddit user comparing a MacBook Pro and a 4x RTX3090 rig running a Q6-quantized Qwen3.6 35B A3B model found that multi-parameter prediction (MTP) speculative decoding actually slowed decode throughput on the multi-GPU setup. Disabling MTP raised decode from about 70 t/s to 82 t/s. The explanation: MTP's draft-and-verify step requires extra synchronization across GPUs, adding overhead that exceeds the gains from accepted draft tokens in layer-split configurations.
A Reddit user comparing a MacBook Pro and a 4x RTX3090 rig running a Q6-quantized Qwen3.6 35B A3B model found that multi-parameter prediction (MTP) speculative decoding actually slowed decode throughput on the multi-GPU setup. Disabling MTP raised decode from about 70 t/s to 82 t/s. The explanation: MTP's draft-and-verify step requires extra synchronization across GPUs, adding overhead that exceeds the gains from accepted draft tokens in layer-split configurations.
10 engagement1 sourceredditFri, Oct 16, 2026, 04:53 PM