Releases·Tool Release
MoEspresso 3 runs Qwen3.8-Flash-Next (125B) at 12-15 tok/s on a 2021 32GB M1 Max, scoring 84.3% on a 48-question benchmark
The author of MoEspresso released version 3 of their homegrown inference engine, which runs the 125B Qwen3.8-Flash-Next on a 2021 M1 Max with 32 GB unified memory at 12-15 decode tokens per second. The key technique is Cache-Prior, which biases decode toward experts already resident in memory, paired with IQ2_K quantization and SSD streaming when experts exceed memory. In the author's 48-question benchmark, the locally run model scores 84.3%, near hosted frontier models.
27 engagement1 sourceSun, Sep 27, 2026, 05:53 PM
AI Overview
- MoEspresso 3 biases already-resident MoE experts during decode via a Cache-Prior scheme; prefill is unbiased.
- The engine uses ik_llama.cpp formats with Metal execution through the mlx-iqk library, mostly IQ2_K quantization for routed projections.
- KVarN K4/V4 keeps RMS error low while leaving more memory for resident experts as context grows.
- MoEspresso 3 ships via Homebrew ('brew install steadfastgaze/tap/moespresso'); the author plans Linux and AMD (Strix Halo) support.
Related
Releases·Tool Release
Faster than Light in Air: 8-22 tg/s Qwen3.8-Flash-Next (Q4/Q4ish) on a 32GB M4 MacBook Air
A developer released Cherenkov, an open-source Apple Silicon inference engine that runs Qwen3.8-Flash-Next on a 32GB M4 MacBook Air at 8-22 tokens per second using only ~21GB of memory. It keeps a bounded working set of experts in unified memory and prefetches upcoming experts via SSD reads, claiming a record for memory-constrained inference.
A developer released Cherenkov, an open-source Apple Silicon inference engine that runs Qwen3.8-Flash-Next on a 32GB M4 MacBook Air at 8-22 tokens per second using only ~21GB of memory. It keeps a bounded working set of experts in unified memory and prefetches upcoming experts via SSD reads, claiming a record for memory-constrained inference.
61 engagement1 sourceredditThu, Sep 10, 2026, 09:31 PM
Social·Community
Qwen3.8-Flash-Next breaks 94% on cupel benchmark in early local tests
A developer reports early local testing of Qwen3.8-Flash-Next on an M4 Max 128GB Studio, achieving over 94% on the cupel benchmark—the first model this year to do so. The model uses the qwen4_exp architecture, which is not yet fully supported by oMLX, requiring workarounds.
A developer reports early local testing of Qwen3.8-Flash-Next on an M4 Max 128GB Studio, achieving over 94% on the cupel benchmark—the first model this year to do so. The model uses the qwen4_exp architecture, which is not yet fully supported by oMLX, requiring workarounds.
195 engagement1 sourceredditThu, Aug 27, 2026, 12:34 PM
Products·launched
Slotstream: run Qwen3.8-Flash-Next 4-bit on low-memory Macs via expert offloading
Slotstream is a Mac-native tool for running the Qwen3.8-Flash-Next 125B-parameter model locally on machines with as little as 16GB of memory. It uses expert offloading and SSD streaming to avoid needing 100GB+ of RAM, packaged as an easy-to-install app built on MLX and Swift. It targets Mac users who want to run large open models locally without expensive hardware.
Slotstream is a Mac-native tool for running the Qwen3.8-Flash-Next 125B-parameter model locally on machines with as little as 16GB of memory. It uses expert offloading and SSD streaming to avoid needing 100GB+ of RAM, packaged as an easy-to-install app built on MLX and Swift. It targets Mac users who want to run large open models locally without expensive hardware.
1 engagement1 sourcehackernewsMon, Aug 31, 2026, 02:46 PM
Developments·Benchmark
Community benchmark tests Qwen3.8-Flash-Next NVFP4 on local agentic coding
A community benchmark run compares the NVFP4-quantized Qwen3.8-Flash-Next against 27B-class and other local models on agentic coding tasks. Results are published on a public local-LLM benchmark site, with a companion post on r/BlackwellPerformance covering single-RTX Pro 6000 performance. It matters to practitioners choosing quantized models for local coding agents.
A community benchmark run compares the NVFP4-quantized Qwen3.8-Flash-Next against 27B-class and other local models on agentic coding tasks. Results are published on a public local-LLM benchmark site, with a companion post on r/BlackwellPerformance covering single-RTX Pro 6000 performance. It matters to practitioners choosing quantized models for local coding agents.
156 engagement2 sourcesredditFri, Aug 28, 2026, 05:30 PM
Products·launched
Qwen3.8-Flash-Next quantization recipe for single GB10/DGX Spark using Intel AutoRound and vLLM
A community recipe and repo for running Qwen3.8-Flash-Next on a single DGX Spark (GB10) device. It combines Intel AutoRound int4 quantization with additional uncalibrated int8/fp8 quantization of specific layers, served via vLLM with an fp8 ngram table offloaded to SSD or an RDMA server. It lets local-LLM hobbyists fit the model on constrained hardware at usable speeds.
A community recipe and repo for running Qwen3.8-Flash-Next on a single DGX Spark (GB10) device. It combines Intel AutoRound int4 quantization with additional uncalibrated int8/fp8 quantization of specific layers, served via vLLM with an fp8 ngram table offloaded to SSD or an RDMA server. It lets local-LLM hobbyists fit the model on constrained hardware at usable speeds.
2 engagement1 sourceredditMon, Aug 31, 2026, 02:04 PM