Run Qwen3.8+Flash-Next and tiny models on Apple Silicon up to 3x faster with MLX.fast fork ishizuki

Releases·Tool Release

mlx-dspark v0.10.0 speeds Qwen3.8-27B up to 3× on Apple Silicon

mlx-dspark, an MLX port of DeepSeek's DSpark speculative-decoding drafters, now supports Qwen3.8-27B via RadixArk's drafter, achieving up to ~3× speedups on Apple Silicon. Benchmarks on an M4 Pro show 2.45× mean speedup for 8-bit targets, with code runs hitting 3.18×.

mlx-dspark, an MLX port of DeepSeek's DSpark speculative-decoding drafters, now supports Qwen3.8-27B via RadixArk's drafter, achieving up to ~3× speedups on Apple Silicon. Benchmarks on an M4 Pro show 2.45× mean speedup for 8-bit targets, with code runs hitting 3.18×.

Releases·Model Release

Community prepares for Qwen 3.8 release, expects MTP gains on Strix Halo

Ahead of the Qwen 3.8 model drop, a Reddit thread discusses how Multi-Token Prediction (MTP) will boost inference speed on AMD Strix Halo hardware, based on experience with Qwen 3.6. Users are sharing software stack tips to maximize performance.

Ahead of the Qwen 3.8 model drop, a Reddit thread discusses how Multi-Token Prediction (MTP) will boost inference speed on AMD Strix Halo hardware, based on experience with Qwen 3.6. Users are sharing software stack tips to maximize performance.

Developments·Hardware

First M5 Ultra benchmarks: 50 tok/s t g, 1800 tok/s pp on Qwen 3 8 27B q4

User-reported benchmarks for Apple's M5 Ultra chip surfaced on the omlx.ai benchmark site, showing strong prefill and token generation speeds for local LLM inference. If representative, the numbers suggest a substantial memory-bandwidth uplift relevant to developers running large quantized models locally.

User-reported benchmarks for Apple's M5 Ultra chip surfaced on the omlx.ai benchmark site, showing strong prefill and token generation speeds for local LLM inference. If representative, the numbers suggest a substantial memory-bandwidth uplift relevant to developers running large quantized models locally.

Qwen announced that Qwen3.8-Flash can now run up to 1.7× faster locally using MTP, with GGUF builds reaching 170 tokens/s on an RTX PRO 6000. The team released GGUF files and a guide, claiming no accuracy loss. For practitioners running local inference, this meaningfully raises achievable throughput on single-GPU setups.

Social·Community

Users benchmark MTPLX multi-token prediction for ~2x faster Qwen3.8 27B inference on Apple Silicon

Community benchmarks show the MTPLX multi-token-prediction runtime roughly doubling local decode speed for Qwen3.8 27B on Apple Silicon Macs, with gains varying sharply by context length. A separate M4 Max test across five runtime/quant stacks found MTPLX fastest up to ~128K context but collapsing at the 256K ceiling, where plain MLX stacks hold up better. Practitioners running local inference on Macs get concrete speed numbers to pick runtimes and quants by context length.

Community benchmarks show the MTPLX multi-token-prediction runtime roughly doubling local decode speed for Qwen3.8 27B on Apple Silicon Macs, with gains varying sharply by context length. A separate M4 Max test across five runtime/quant stacks found MTPLX fastest up to ~128K context but collapsing at the 256K ceiling, where plain MLX stacks hold up better. Practitioners running local inference on Macs get concrete speed numbers to pick runtimes and quants by context length.