The UnbiasedStrata project published a self-contained inference engine combining Strata's engine with llama.cpp's tokenizer, chat template, and HTTP code in one C++ binary. It serves Qwen3.8-Flash-Next behind an OpenAI-style API with no network access needed to build or run, targeting developers who want a dependency-free local deployment.
Two community inference engines now run Qwen3.8-Flash-Next, a 125B MoE model, on consumer NVIDIA GPUs. Speedy, a benchmarked fork of Strata, reaches 229K native context (1M tier) on RTX 3080 10 GB cards, while RustingStrata, a pure-Rust engine, roughly doubles the original Strata's decode speed on a 12 GB RTX 3060. Both demonstrate that large MoE models can run on consumer hardware via hybrid CPU+GPU expert placement.
Two community inference engines now run Qwen3.8-Flash-Next, a 125B MoE model, on consumer NVIDIA GPUs. Speedy, a benchmarked fork of Strata, reaches 229K native context (1M tier) on RTX 3080 10 GB cards, while RustingStrata, a pure-Rust engine, roughly doubles the original Strata's decode speed on a 12 GB RTX 3060. Both demonstrate that large MoE models can run on consumer hardware via hybrid CPU+GPU expert placement.
A Reddit user shared a working Strata configuration and benchmarks for running Qwen3.8-Flash-Next (IQ3_S quant) on an RTX 3070 with a Ryzen 5 5600 and 64GB RAM, achieving roughly 1000 tps prefill and 40 tps decode at 262k max context. The post details which environment flags delivered the biggest gains and invites tuning suggestions.
A Reddit user shared a working Strata configuration and benchmarks for running Qwen3.8-Flash-Next (IQ3_S quant) on an RTX 3070 with a Ryzen 5 5600 and 64GB RAM, achieving roughly 1000 tps prefill and 40 tps decode at 262k max context. The post details which environment flags delivered the biggest gains and invites tuning suggestions.
Local-LLaMA users are getting usable throughput from Qwen3.8-Flash-Next on consumer hardware with self-built vLLM modifications. One user's Claude-built fork with custom RDNA2 kernels runs the model on eight used Radeon Pro V620s ($2800, 256 GB VRAM) at 60-100 t/s decode and 3000+ t/s prefill. Another posted fixes letting six RTX 3090s serve 7 users at 262k contexts via pipeline parallelism, int8 KV cache, and SSD-offloaded n-gram tables.
Local-LLaMA users are getting usable throughput from Qwen3.8-Flash-Next on consumer hardware with self-built vLLM modifications. One user's Claude-built fork with custom RDNA2 kernels runs the model on eight used Radeon Pro V620s ($2800, 256 GB VRAM) at 60-100 t/s decode and 3000+ t/s prefill. Another posted fixes letting six RTX 3090s serve 7 users at 262k contexts via pipeline parallelism, int8 KV cache, and SSD-offloaded n-gram tables.
Developer Niko1221 released Strata v0.1.40 with official support for running Qwen3.8-Flash-Next on Strix Halo machines, claiming best-in-class long-context decode and prompt throughput using Unsloth Q4 and GSQ-RCO weights. It supports up to 1M context with little speed loss, but support is experimental and Linux-only.
Developer Niko1221 released Strata v0.1.40 with official support for running Qwen3.8-Flash-Next on Strix Halo machines, claiming best-in-class long-context decode and prompt throughput using Unsloth Q4 and GSQ-RCO weights. It supports up to 1M context with little speed loss, but support is experimental and Linux-only.
A Reddit user with an AMD R9700 plus RTX 5060 Ti benchmarked Qwen3.8-Flash-Next at an IQ3_XXS quant (~70GB) using llama.cpp: 21 t/s on system RAM plus the R9700, versus ~60 t/s under Strata. The speedup makes the model usable as a daily driver, but loading it in Strata consumes 96% of the user's 64GB system RAM, forcing a choice between LLM inference and ComfyUI video workloads.
A Reddit user with an AMD R9700 plus RTX 5060 Ti benchmarked Qwen3.8-Flash-Next at an IQ3_XXS quant (~70GB) using llama.cpp: 21 t/s on system RAM plus the R9700, versus ~60 t/s under Strata. The speedup makes the model usable as a daily driver, but loading it in Strata consumes 96% of the user's 64GB system RAM, forcing a choice between LLM inference and ComfyUI video workloads.