SiliconSpecies ships Swift-Qwen3.8-27B 4-bit package for Mac-only Splash engine, hitting 87.8 tok/s at 64k context on M5 Max

Releases·Tool Release

Qwen3.8-27B at 144 tok/s on an M5 Max MacBook Pro

Inco released Splash, an open-source inference engine built for Apple silicon, running Qwen3.8-27B at up to 144 tok/s on an M5 Max MacBook Pro. It claims up to 3× Ollama's decode speed and 2× oMLX, with nearly 4× gains when agents fan out into sub-agents. It ships as a CLI server plus an LM Studio runtime, targeting local agent work.

Inco released Splash, an open-source inference engine built for Apple silicon, running Qwen3.8-27B at up to 144 tok/s on an M5 Max MacBook Pro. It claims up to 3× Ollama's decode speed and 2× oMLX, with nearly 4× gains when agents fan out into sub-agents. It ships as a CLI server plus an LM Studio runtime, targeting local agent work.

Social·Community

Swift-1.5-Qwen3.8-27b-oQ8e-mtp hits 34.8 tok/s on Apple M5 Max, averaging 51k tokens per llm-bench.io run vs Qwen3.8's 77k

A user ran Swift-1.5-Qwen3.8-27b-oQ8e-mtp through llm-bench.io on an Apple M5 Max (64 GB, oMLX, thinking at xhigh) and compared it to base Qwen3.8 27B. Swift 1.5 generates far fewer tokens, cutting full benchmark runs from ~38 to ~24 minutes, while quality judged as a tie (85.8 vs 85.0). The user plans to try it as a daily driver.

A user ran Swift-1.5-Qwen3.8-27b-oQ8e-mtp through llm-bench.io on an Apple M5 Max (64 GB, oMLX, thinking at xhigh) and compared it to base Qwen3.8 27B. Swift 1.5 generates far fewer tokens, cutting full benchmark runs from ~38 to ~24 minutes, while quality judged as a tie (85.8 vs 85.0). The user plans to try it as a daily driver.

A Reddit user tested four Apple Silicon local inference engines on a new 14" M3 Max MacBook Pro (36 GB) with Qwen3.8-27B-4bit and Qwen3.6-35B-A3B-4bit. Rapid-MLX was fastest on the MoE model (~110-118 tok/s) while Splash edged the dense model (~32 tok/s). The 125B Qwen3.8-Flash-Next was deemed impractical on 36 GB, needing 96 GB+.

Social·Community

Users benchmark MTPLX multi-token prediction for ~2x faster Qwen3.8 27B inference on Apple Silicon

Community benchmarks show the MTPLX multi-token-prediction runtime roughly doubling local decode speed for Qwen3.8 27B on Apple Silicon Macs, with gains varying sharply by context length. A separate M4 Max test across five runtime/quant stacks found MTPLX fastest up to ~128K context but collapsing at the 256K ceiling, where plain MLX stacks hold up better. Practitioners running local inference on Macs get concrete speed numbers to pick runtimes and quants by context length.

Community benchmarks show the MTPLX multi-token-prediction runtime roughly doubling local decode speed for Qwen3.8 27B on Apple Silicon Macs, with gains varying sharply by context length. A separate M4 Max test across five runtime/quant stacks found MTPLX fastest up to ~128K context but collapsing at the 256K ceiling, where plain MLX stacks hold up better. Practitioners running local inference on Macs get concrete speed numbers to pick runtimes and quants by context length.

A developer found Splash's default kernel rules were measured only on 16/20-core M5s and a 32-core M4 Max, so a 40-core M5 Max runs suboptimal guesses. Using Splash's built-in tune-kernels developer tool to benchmark every quantized matrix-multiply layout on their machine yielded roughly 20% faster decode, with split-K layouts winning for the 8-row draft-token step.