Developer updates gfx906 llama.cpp fork for Radeon VII / MI50 / MI60 with +14% PP and adaptive Flash Attention

Social·Community

Reddit user asks which llama.cpp fork best optimizes AMD MI50 GPUs, citing mx-llama.cpp and exabit-io forks

A Reddit user with an AMD Instinct MI50 asks which llama.cpp fork best squeezes performance from the card, citing two candidate forks: mxxm-t/mx-llama.cpp and exabit-io/llama.cpp. The thread reflects ongoing community interest in optimizing inference on older AMD data-center GPUs via unofficial forks.

A Reddit user with an AMD Instinct MI50 asks which llama.cpp fork best squeezes performance from the card, citing two candidate forks: mxxm-t/mx-llama.cpp and exabit-io/llama.cpp. The thread reflects ongoing community interest in optimizing inference on older AMD data-center GPUs via unofficial forks.

Releases·Tool Release

dual 7900 xtx - some guy made a pretty optimized fork of lamacpp optimized for this setup Qwen 3.8 Q8 at 82 tokens / seconds decode

A community-optimized llama.cpp fork targeting dual AMD Radeon 7900 XTX setups boosts Qwen 3 8B Q8 decode speed from ~28 tok/s (vanilla llama.cpp with Vulkan) to 82 tok/s. A Reddit user shared the GitHub repo, reporting stable results at 60k context, which could interest practitioners running local inference on consumer AMD hardware.

A community-optimized llama.cpp fork targeting dual AMD Radeon 7900 XTX setups boosts Qwen 3 8B Q8 decode speed from ~28 tok/s (vanilla llama.cpp with Vulkan) to 82 tok/s. A Reddit user shared the GitHub repo, reporting stable results at 60k context, which could interest practitioners running local inference on consumer AMD hardware.

Releases·Tool Release

2x Tesla p100s, q6_k quant, Qwen 3.8 27B ~60tps V3.0

A developer released V3.0 of llama.cpp kernel optimizations for Tesla P100 GPUs, lifting Qwen 3.8 27B Q6_K decode from 7-15 tps to 50-60 tps on a two-card $160 setup. The work, largely AI-assisted, was merged upstream on Sept 22 with strict regression testing.

A developer released V3.0 of llama.cpp kernel optimizations for Tesla P100 GPUs, lifting Qwen 3.8 27B Q6_K decode from 7-15 tps to 50-60 tps on a two-card $160 setup. The work, largely AI-assisted, was merged upstream on Sept 22 with strict regression testing.

Releases·Tool Release

llama.cpp Metal PR #29869 adds few-row MMA mat-mul and batched copies, boosting DFlash2 speculative decoding up to 4x

A llama.cpp pull request by pratiknarola-t adds Metal few-row MMA matmul kernels and batched copies to accelerate speculative decoding on Apple GPUs. Benchmarks with a Qwen3.8-27B DFlash2 setup on llama-server show decode throughput rising from 16.8 tok/s to 110.0 tok/s on code prompts with no batch dimension. The work targets Apple Metal users running speculative decoding with draft models.

A llama.cpp pull request by pratiknarola-t adds Metal few-row MMA matmul kernels and batched copies to accelerate speculative decoding on Apple GPUs. Benchmarks with a Qwen3.8-27B DFlash2 setup on llama-server show decode throughput rising from 16.8 tok/s to 110.0 tok/s on code prompts with no batch dimension. The work targets Apple Metal users running speculative decoding with draft models.

Developments·Hardware

First M5 Ultra benchmarks: 50 tok/s t g, 1800 tok/s pp on Qwen 3 8 27B q4

User-reported benchmarks for Apple's M5 Ultra chip surfaced on the omlx.ai benchmark site, showing strong prefill and token generation speeds for local LLM inference. If representative, the numbers suggest a substantial memory-bandwidth uplift relevant to developers running large quantized models locally.

User-reported benchmarks for Apple's M5 Ultra chip surfaced on the omlx.ai benchmark site, showing strong prefill and token generation speeds for local LLM inference. If representative, the numbers suggest a substantial memory-bandwidth uplift relevant to developers running large quantized models locally.