Releases·Tool Release
Splash fork optimised for M5 Max: ~1.5× faster (1.25× single request)
A developer used Opus 5.5 to fork Inco's Splash engine, optimising it for Apple's 40-core M5 Max. The fork, dubbed Splish, is roughly 1.25× faster at a single request and up to 1.5× faster at 2–4 concurrent requests, with unchanged output quality. Optimisations include M5-specific kernel tuning, new verify kernels for the M5's tensor units, and a coding-agent copy rule.
39 engagement1 sourceSun, Sep 27, 2026, 06:40 AM
AI Overview
- 'Splish' fork of Inco's Splash engine, built with Opus 5.5, runs 4-bit models up to 1.5× faster at 2–4 requests and 1.25× at one request on a 40-core M5 Max
- M5 Max–measured kernel choices via Splash's own tuner lift Swift-1.5 from 74.7 to 89.8 tok/s (+20%), loadable via SPLASH_KERNEL_CHOICES without rebuilding
- New verify kernels for M5 tensor units add +5% at 1 request and +10–19% at 2–4, making 2/3/4-request decode steps 1.32×/1.49×/1.36× faster than tuned Splash
- Tuned Qwen3.6-35B-A3B with the new kernels gains +5%/+18%/+22%/+18% at 1–4 requests; GGUF Q8_0/Q4_K_M/Q6_K loads gain +2–14% per kernel with bit-identical output
- TensorFold-style copy rule for coding agents: drafts reuse seen text verbatim, speeding whole-file edits +24–42% while other workloads stay within ±3%
- Real-world Deepseek Harness short-story prompts rise from 45–51 tok/s to 56–64 tok/s; tuned settings apply only to 40-core M5 Max, with an auto-tuner planned
Related
Social·Community
Splash on a 40-core M5 Max: +20% decode by tuning the kernels for your own chip
A developer found Splash's default kernel rules were measured only on 16/20-core M5s and a 32-core M4 Max, so a 40-core M5 Max runs suboptimal guesses. Using Splash's built-in tune-kernels developer tool to benchmark every quantized matrix-multiply layout on their machine yielded roughly 20% faster decode, with split-K layouts winning for the 8-row draft-token step.
A developer found Splash's default kernel rules were measured only on 16/20-core M5s and a 32-core M4 Max, so a 40-core M5 Max runs suboptimal guesses. Using Splash's built-in tune-kernels developer tool to benchmark every quantized matrix-multiply layout on their machine yielded roughly 20% faster decode, with split-K layouts winning for the 8-row draft-token step.
24 engagement1 sourceredditFri, Sep 25, 2026, 01:13 PM
Social·Community
oMLX vs Rapid-MLX vs Splash vs MTPLX on M3 Max 36 GB: 110 tok/s on Qwen3.6-35B-A3B, ~32 tok/s on Qwen3.8-27B
A Reddit user tested four Apple Silicon local inference engines on a new 14" M3 Max MacBook Pro (36 GB) with Qwen3.8-27B-4bit and Qwen3.6-35B-A3B-4bit. Rapid-MLX was fastest on the MoE model (~110-118 tok/s) while Splash edged the dense model (~32 tok/s). The 125B Qwen3.8-Flash-Next was deemed impractical on 36 GB, needing 96 GB+.
A Reddit user tested four Apple Silicon local inference engines on a new 14" M3 Max MacBook Pro (36 GB) with Qwen3.8-27B-4bit and Qwen3.6-35B-A3B-4bit. Rapid-MLX was fastest on the MoE model (~110-118 tok/s) while Splash edged the dense model (~32 tok/s). The 125B Qwen3.8-Flash-Next was deemed impractical on 36 GB, needing 96 GB+.
18 engagement1 sourceredditMon, Oct 5, 2026, 03:11 PM
Releases·Tool Release
Lithos Metal: open-source Metal megakernel inference engine brings low-latency LLM serving to Apple silicon Macs
Lithos AI released lithos-metal, a fully open-source LLM inference engine that generates Metal megakernels for Apple silicon Macs, with local serving for coding agents. It combines layer-wise mixer megakernels, GPU-resident generation control, and DSpark speculative decoding to cut overhead and improve target-model pass utilization. It matters to practitioners running low-latency local LLM inference and coding agents on Macs.
Lithos AI released lithos-metal, a fully open-source LLM inference engine that generates Metal megakernels for Apple silicon Macs, with local serving for coding agents. It combines layer-wise mixer megakernels, GPU-resident generation control, and DSpark speculative decoding to cut overhead and improve target-model pass utilization. It matters to practitioners running low-latency local LLM inference and coding agents on Macs.
81 engagement1 sourcegithubMon, Oct 5, 2026, 03:23 PM
Social·Community
Users run GLM 5.2 (744B MoE) locally on consumer hardware with CPU-only inference and Flash MOE
Multiple users report running GLM 5.2, a 744B-parameter mixture-of-experts model, on consumer-grade hardware. One user achieved 2-2.8 tokens/s on a MacBook Pro M5 with 48 GB RAM using Flash MOE. Another ran CPU-only inference on an AMD EPYC 9654 with 768 GB RAM, getting 5-9 t/s decode. A third used the colibrì engine to stream experts from disk on a 132 GB RAM box, achieving ~0.03 tok/s cold start. These results show that large MoE models can be run locally with optimization, though performance varies widely.
Multiple users report running GLM 5.2, a 744B-parameter mixture-of-experts model, on consumer-grade hardware. One user achieved 2-2.8 tokens/s on a MacBook Pro M5 with 48 GB RAM using Flash MOE. Another ran CPU-only inference on an AMD EPYC 9654 with 768 GB RAM, getting 5-9 t/s decode. A third used the colibrì engine to stream experts from disk on a 132 GB RAM box, achieving ~0.03 tok/s cold start. These results show that large MoE models can be run locally with optimization, though performance varies widely.
Updated Sat, Jul 18, 2026, 09:38 PM — A user reports running GLM 5.2 on a MacBook Pro M5 with Flash MOE at 2-2.8 t/s, while another achieves 5-9 t/s decode on
110169 engagement3 sourcesredditMon, Jul 13, 2026, 07:26 PM
Releases·Tool Release
mlx-dspark v0.10.0 speeds Qwen3.8-27B up to 3× on Apple Silicon
mlx-dspark, an MLX port of DeepSeek's DSpark speculative-decoding drafters, now supports Qwen3.8-27B via RadixArk's drafter, achieving up to ~3× speedups on Apple Silicon. Benchmarks on an M4 Pro show 2.45× mean speedup for 8-bit targets, with code runs hitting 3.18×.
mlx-dspark, an MLX port of DeepSeek's DSpark speculative-decoding drafters, now supports Qwen3.8-27B via RadixArk's drafter, achieving up to ~3× speedups on Apple Silicon. Benchmarks on an M4 Pro show 2.45× mean speedup for 8-bit targets, with code runs hitting 3.18×.
210 engagement1 sourceredditFri, Aug 14, 2026, 09:54 PM