Releases·Tool Release
LlamAmpere v0.4 hits 95+ TPS through 100K generated for Qwen3.8 27B at 262K ctx on a single 3090
Developer JakeATX released LlamAmpere v0.4, a Llama.cpp fork with Ampere-specific optimizations, achieving 95+ tokens/second over 100K generated tokens for a Qwen3.8 27B model at 262K context on one RTX 3090. The update delivers ~10% speed gains and 10%+ more max context versus the prior version, with vLLM as the closest competing runtime. The author also quantifies the quality cost of the TQ5/TQ4 KV cache quantization used.
57 engagement1 sourceMon, Sep 28, 2026, 06:35 PM
AI Overview
- LlamAmpere v0.4, a Llama.cpp fork with Ampere-specific improvements, improves speed ~10% and max context 10%+ over the prior version on a 4.6bpw model
- Developer JakeATX reports TQ5/TQ4 KV-cache quantization shows under 1/3 the KLD of moving from 8-bit to 4.6-bit weights, with no statistically significant task-level difference vs 8/8 KV
- v0.4's closest competition on the 3090 comes from vLLM, per JakeATX
- v0.5 will target 12GB cards with generation-wide speedups and add KVaRN using a new codec, pending further testing
Related
Releases·Tool Release
LlamAmpere update runs Qwen3.8 27B with 200K+ ctx and MTP on 12GB Ampere cards
A developer released a new LlamAmpere version focused on running Qwen3.8 27B models on 12GB Ampere cards, adding Staged + Journaled KVaRN, compact MTP caches, and quantization options that fit 205k–230k context at 11GB. The quantized fusion models retain 85% of BF16 LiveCode Bench performance and reach ~65–70 tps on 3080/ti via MTP. The runtime and models are MIT-licensed on GitHub and Hugging Face.
A developer released a new LlamAmpere version focused on running Qwen3.8 27B models on 12GB Ampere cards, adding Staged + Journaled KVaRN, compact MTP caches, and quantization options that fit 205k–230k context at 11GB. The quantized fusion models retain 85% of BF16 LiveCode Bench performance and reach ~65–70 tps on 3080/ti via MTP. The runtime and models are MIT-licensed on GitHub and Hugging Face.
74 engagement1 sourceredditFri, Oct 9, 2026, 09:39 PM
Releases·Tool Release
llamAmpere: llama.cpp fork claims 90+ TPS on RTX 3090 for 27B models, up to 240K context
A developer released llamAmpere, a custom llama.cpp fork optimized for Ampere-architecture GPUs, alongside a Qwen3.8-27B GGUF quant on Hugging Face. The recommended configuration reaches 90+ tokens/second through 100K context (up to 240K) on a single RTX 3090, which the author says exceeds API speeds for 27B models and significantly improves local LLM utility on 30-series cards.
A developer released llamAmpere, a custom llama.cpp fork optimized for Ampere-architecture GPUs, alongside a Qwen3.8-27B GGUF quant on Hugging Face. The recommended configuration reaches 90+ tokens/second through 100K context (up to 240K) on a single RTX 3090, which the author says exceeds API speeds for 27B models and significantly improves local LLM utility on 30-series cards.
182 engagement2 sourcesredditTue, Sep 15, 2026, 01:32 AM
Releases·Tool Release
UPDATE: Qwen 3.8 27B 140 tok/s on single RTX 3090 Megakernel: KL divergence 0.0009 vs llama.cpp
A developer posted accuracy and speed benchmarks for an open-source CUDA megakernel engine (part of the open-jet repo) that runs Qwen3.8-27B 1.4-1.9x faster than llama.cpp on a single RTX 3090. The update answers accuracy questions with KL-divergence comparisons and lays out quantization support, hardware limits, and contribution asks. It matters to local-inference practitioners as a tuned single-GPU alternative to llama.cpp for this model.
A developer posted accuracy and speed benchmarks for an open-source CUDA megakernel engine (part of the open-jet repo) that runs Qwen3.8-27B 1.4-1.9x faster than llama.cpp on a single RTX 3090. The update answers accuracy questions with KL-divergence comparisons and lays out quantization support, hardware limits, and contribution asks. It matters to local-inference practitioners as a tuned single-GPU alternative to llama.cpp for this model.
1 engagement1 sourceredditSun, Oct 11, 2026, 01:04 PM
Releases·Tool Release
Qwen3.8 27B on one AMD R9700: 569,878-token reusable cache, 262K context, ~180 tok/s via W3A4 3-bit weights
Eliovp BV published a new coding mode for its Paiton vLLM plugin that runs Qwen3.8 27B on a single AMD Radeon AI PRO R9700 (32 GB) with 262,144-token context per request and 569,878 tokens of reusable prefix cache. Coding-agent turns start up to 12× sooner and full sessions finish ~6× faster because only new tokens are re-prefilled. The release pairs a selective 3-bit weight quantization scheme with unchanged MXFP4 as the more accurate option.
Eliovp BV published a new coding mode for its Paiton vLLM plugin that runs Qwen3.8 27B on a single AMD Radeon AI PRO R9700 (32 GB) with 262,144-token context per request and 569,878 tokens of reusable prefix cache. Coding-agent turns start up to 12× sooner and full sessions finish ~6× faster because only new tokens are re-prefilled. The release pairs a selective 3-bit weight quantization scheme with unchanged MXFP4 as the more accurate option.
11 engagement1 sourceredditSun, Oct 4, 2026, 01:12 PM
Social·Community
Running the uncensored Qwen3.8-27B (HauhauCS) on a 4090 at 262K context and ~130 tok/s
A Reddit user reports running HauhauCS's uncensored Qwen3.8-27B on an RTX 4090 using NInfer, a C++/CUDA inference engine, after converting the GGUF-only release to NInfer's own format. The setup reaches the model's full native 262K context at ~130 tok/s decode, up from 91.6 tok/s / 131K under llama.cpp, with perplexity within 1.3% of the official artifact.
A Reddit user reports running HauhauCS's uncensored Qwen3.8-27B on an RTX 4090 using NInfer, a C++/CUDA inference engine, after converting the GGUF-only release to NInfer's own format. The setup reaches the model's full native 262K context at ~130 tok/s decode, up from 91.6 tok/s / 131K under llama.cpp, with perplexity within 1.3% of the official artifact.
45 engagement1 sourceredditFri, Oct 9, 2026, 05:27 AM