2x Tesla p100s, q6_k quant, Qwen 3.8 27B ~60tps V3.0

Releases·Tool Release

Developer updates gfx906 llama.cpp fork for Radeon VII / MI50 / MI60 with +14% PP and adaptive Flash Attention

A developer updated their gfx906 llama.cpp fork for AMD Radeon VII / MI50 / MI60 GPUs after earlier criticism over insufficient benchmarks. Reworking the small-Q Flash Attention path and adding adaptive native/convert selection yielded +14.1% first-batch PP and +9% long-context fill vs upstream llama.cpp mainline. The update matters to practitioners running local inference on older AMD CDNA/GFX9 hardware, where DFlash2 adoption on a Qwen 3.8 27B production setup had made some prior optimizations regress.

A developer updated their gfx906 llama.cpp fork for AMD Radeon VII / MI50 / MI60 GPUs after earlier criticism over insufficient benchmarks. Reworking the small-Q Flash Attention path and adding adaptive native/convert selection yielded +14.1% first-batch PP and +9% long-context fill vs upstream llama.cpp mainline. The update matters to practitioners running local inference on older AMD CDNA/GFX9 hardware, where DFlash2 adoption on a Qwen 3.8 27B production setup had made some prior optimizations regress.

Releases·Tool Release

PXA ships as set-and-forget LLM engine for Pascal/Volta GPUs with PXQ codec, auto-tuning, 2.3× prefill gain

The PXA project released v2026.09.07-rc1, an auto-tuned LLM inference engine targeting Pascal and Volta GPUs. It uses a PXQ codec plus per-card tuned kernels and claims 2.3× the long-prompt prefill speed of the next-best Volta engine on that engine's own hardware. The release demonstrates a 177B-class MoE model with 150k context running on four Tesla P100s.

The PXA project released v2026.09.07-rc1, an auto-tuned LLM inference engine targeting Pascal and Volta GPUs. It uses a PXQ codec plus per-card tuned kernels and claims 2.3× the long-prompt prefill speed of the next-best Volta engine on that engine's own hardware. The release demonstrates a 177B-class MoE model with 150k context running on four Tesla P100s.

Social·Community

User benchmarks Inco AI's DFlash 2 speculative decoding on Qwen 3.8 27B, reports 2.26x speedup

A developer benchmarked Inco AI's DFlash 2 speculative decoding method in llama.cpp on Qwen 3.8 27B, reporting a 2.26x speedup on 100 LiveCodeBench coding prompts. The method also showed up to 4.68x when combined with an n-gram drafter, with results varying by task.

A developer benchmarked Inco AI's DFlash 2 speculative decoding method in llama.cpp on Qwen 3.8 27B, reporting a 2.26x speedup on 100 LiveCodeBench coding prompts. The method also showed up to 4.68x when combined with an n-gram drafter, with results varying by task.

Social·Community

User benchmarks llama.cpp RPC with 5070 Ti and 1080 Ti over gigabit ethernet

A user tested llama.cpp's RPC feature with a 5070 Ti and a 1080 Ti over gigabit ethernet, finding it functional but requiring a trade-off between prefill and generation speed. Using UD-Q4_K_XL with q8 KV cache and 96k context, they achieved 350 pp/s and 36 tg/s when prioritizing generation, or 560 pp/s and 19 tg/s when prioritizing prefill. The goal was to run Qwen 3.8 27B without compromising KV cache or quant.

A user tested llama.cpp's RPC feature with a 5070 Ti and a 1080 Ti over gigabit ethernet, finding it functional but requiring a trade-off between prefill and generation speed. Using UD-Q4_K_XL with q8 KV cache and 96k context, they achieved 350 pp/s and 36 tg/s when prioritizing generation, or 560 pp/s and 19 tg/s when prioritizing prefill. The goal was to run Qwen 3.8 27B without compromising KV cache or quant.

Social·Community

Users run Qwen 3.8 27B locally on consumer GPUs: 75 tok/s on 16GB, agentic coding on 12GB, AMD slowdowns

Local LLM users are sharing configurations for running Qwen 3.8 27B on consumer GPUs. An RTX 5080 setup reaches ~75 tok/s decode using a custom hybrid GGUF quant with multi-token prediction, while a 12GB laptop GPU runs it for agentic coding at 9–11 tok/s with Unsloth Dynamic 3.0 quants. An AMD RX 9060 XT user finds the 27B too slow and switches to Ornith 1.5 9B.

Local LLM users are sharing configurations for running Qwen 3.8 27B on consumer GPUs. An RTX 5080 setup reaches ~75 tok/s decode using a custom hybrid GGUF quant with multi-token prediction, while a 12GB laptop GPU runs it for agentic coding at 9–11 tok/s with Unsloth Dynamic 3.0 quants. An AMD RX 9060 XT user finds the 27B too slow and switches to Ornith 1.5 9B.