Releases·Tool Release
Update: Strix Halo + R9700 with llama-halo-hybrid - now beats DGX Spark
A developer updated llama-halo-hybrid, an MIT-licensed llama.cpp modification that splits model layers between an AMD Strix Halo APU and a sidecar Radeon R9700 GPU. The hybrid setup now exceeds 60 tok/s decode and 2000+ tok/s prefill with 256k context, reportedly beating NVIDIA's DGX Spark. The approach targets practitioners running large-context open models on consumer hardware instead of proprietary workstations.
4 engagement1 sourceWed, Sep 30, 2026, 07:46 PM
AI Overview
- llama-halo-hybrid splits dense layers, KV cache, and some layers between Strix Halo APU and an R9700 sidecar GPU connected via PCIe extender, OCuLink, or Thunderbolt
- Open-source MIT-licensed llama.cpp fork breaks 60+ tok/s decode and 2000+ tok/s prefill with full 256k context, no custom quant required
- Outperforms DGX Spark on Qwen-3.8-flash-next and slightly better with Swift-1.5, tested mainly with Q4/Q4_K_XL quants
- Tuned mostly for Qwen and GLM model families; author recommends gufo for Strix Halo-only setups
Related
Social·Community
strix-llama rdna35-perf branch: prompt processing +21–54% over Vulkan and up to 2.7× over stock ROCm at 128k on Strix Halo
A developer shared a 10-commit branch (`rdna35-perf`) on the strix-llama.cpp fork that adds an RDNA3.5 flash attention kernel and other optimizations for the Radeon 8060S (Strix Halo, 128 GB). Benchmarks across Qwen3.5/3.6/3.8 models show prompt processing 21–54% faster than Vulkan and up to 174% faster than stock ROCm at long context, with unchanged perplexity. It matters to practitioners running local inference on Strix Halo, where stock ROCm flash attention degrades sharply at deep contexts.
A developer shared a 10-commit branch (`rdna35-perf`) on the strix-llama.cpp fork that adds an RDNA3.5 flash attention kernel and other optimizations for the Radeon 8060S (Strix Halo, 128 GB). Benchmarks across Qwen3.5/3.6/3.8 models show prompt processing 21–54% faster than Vulkan and up to 174% faster than stock ROCm at long context, with unchanged perplexity. It matters to practitioners running local inference on Strix Halo, where stock ROCm flash attention degrades sharply at deep contexts.
21 engagement1 sourceredditMon, Oct 12, 2026, 04:09 AM
Social·Community
Strix Halo user reports mixed results adding R9700 dGPU via Oculink
A Strix Halo owner added an R9700 dGPU via Oculink and found performance gains limited. The Unsloth Q4_K_XL quant of DSv4 Flash 0731 failed to run, but Minimax M-2.7 IQ4_XS matched APU-only token generation with a 2-3x prompt processing boost.
A Strix Halo owner added an R9700 dGPU via Oculink and found performance gains limited. The Unsloth Q4_K_XL quant of DSv4 Flash 0731 failed to run, but Minimax M-2.7 IQ4_XS matched APU-only token generation with a 2-3x prompt processing boost.
22 engagement1 sourceredditTue, Aug 25, 2026, 04:14 AM
Social·Community
Gufo inference runs Qwen 3.8 Flash Next on Strix Halo at 2x the prompt-processing speed of the fastest llama.cpp fork
A user recommends the open-source Gufo inference engine for running Qwen 3.8 Flash Next on AMD Strix Halo, reporting roughly 2x faster prompt processing than the fastest Strix Halo llama.cpp fork, with even larger gains over mainline llama.cpp at high context. The post links Gufo's own Qwen 3.8 Flash Next benchmarks page, which the user says hold up on real workloads.
A user recommends the open-source Gufo inference engine for running Qwen 3.8 Flash Next on AMD Strix Halo, reporting roughly 2x faster prompt processing than the fastest Strix Halo llama.cpp fork, with even larger gains over mainline llama.cpp at high context. The post links Gufo's own Qwen 3.8 Flash Next benchmarks page, which the user says hold up on real workloads.
113 engagement2 sourcesredditMon, Sep 28, 2026, 07:49 AM
Social·Community
Strix Halo llama.cpp forks hit 80-90% hardware theory vs ~50% on official build
A community member argues official llama.cpp poorly serves AMD Strix Halo (gfx1150), reaching under 50% of theoretical hardware throughput, and shares guidance for optimal local inference setups. The post matters to practitioners running LLMs on Strix Halo's unified-memory silicon.
A community member argues official llama.cpp poorly serves AMD Strix Halo (gfx1150), reaching under 50% of theoretical hardware throughput, and shares guidance for optimal local inference setups. The post matters to practitioners running LLMs on Strix Halo's unified-memory silicon.
172 engagement1 sourceredditTue, Sep 8, 2026, 12:43 AM
Social·Community
Reddit users say hardware shortage pushes Local LLM hobbyists to tweak inference, quantization, and custom llama.cpp forks
A Reddit essay argues the current hardware shortage is driving a renaissance in the Local LLM community, forcing hobbyists to optimize inference engines, quantization, and architectures instead of buying more compute. The author cites forked llama.cpp builds and halogen-flash-server for Strix Halo that roughly doubled decode and multiplied prefill speed for Qwen 3.8 Flash Next, and likens the moment to the early-web era of forum-driven, end-to-end learning.
A Reddit essay argues the current hardware shortage is driving a renaissance in the Local LLM community, forcing hobbyists to optimize inference engines, quantization, and architectures instead of buying more compute. The author cites forked llama.cpp builds and halogen-flash-server for Strix Halo that roughly doubled decode and multiplied prefill speed for Qwen 3.8 Flash Next, and likens the moment to the early-web era of forum-driven, end-to-end learning.
1.1k engagement1 sourceredditSun, Sep 13, 2026, 09:56 AM