Social·Community
Qwen Flash next on 64GB RAM unified iGPU anyone ? (non-mac)
A Reddit user asks whether Qwen Flash Next, reportedly run by others on 12GB VRAM plus 64GB RAM and on 64GB unified-memory Macs, could work on a 64GB RAM mini PC with an AMD 780M iGPU. They currently run 125B Ling 3.0 Flash at Q2 quants via llama.cpp (Vulkan) but note llama.cpp cannot yet offload n-grams to SSD, and other inference engines lack Vulkan support.
23 engagement1 sourceSat, Oct 3, 2026, 01:43 PM
AI Overview
- Reddit user reports running Ling 3.0 flash 125B at Q2 quants via llama.cpp (Vulkan) on a 64GB RAM mini PC with a Radeon 780M iGPU
- User notes llama.cpp cannot offload ngrams to SSD for Qwen Flash Next, and other inference engines lack Vulkan support
Related
Social·Community
Qwen3.8-Flash-Next 177B (UD-IQ3_XXS) hits 11–15 tok/s on one RTX 5070 12GB via llama.cpp expert streaming
A developer benchmarked a llama.cpp-based expert streaming setup for Qwen3.8-Flash-Next 177B (UD-IQ3_XXS) on Windows, reaching roughly 11.5 tok/s in benchmarks and 14–15 tok/s in conversation on an RTX 5070 12GB with 32GB DDR4 — up from about 7 tok/s. The gains came from fixing Windows I/O queue-depth issues, one file handle per worker, and a page-locked hot-expert tier for pulling hot expert weights. The setup and methodology are published on GitHub with demo videos.
A developer benchmarked a llama.cpp-based expert streaming setup for Qwen3.8-Flash-Next 177B (UD-IQ3_XXS) on Windows, reaching roughly 11.5 tok/s in benchmarks and 14–15 tok/s in conversation on an RTX 5070 12GB with 32GB DDR4 — up from about 7 tok/s. The gains came from fixing Windows I/O queue-depth issues, one file handle per worker, and a page-locked hot-expert tier for pulling hot expert weights. The setup and methodology are published on GitHub with demo videos.
81 engagement1 sourceredditSat, Oct 3, 2026, 06:49 AM
Social·Community
Running Qwen3.8-Flash-Next, a 125B MoE model, on one RTX 3090 Ti
Developer runs Qwen3.8-Flash-Next 125B MoE on a single RTX 3090 Ti
Developer runs Qwen3.8-Flash-Next 125B MoE on a single RTX 3090 Ti
0 engagement1 sourcerssSun, Oct 4, 2026, 01:25 AM
Social·Community
POV 8GB VRAM + 16GB RAM folks checking locallama everyday:
A LocalLLaMA thread captures low-VRAM hobbyists (8GB VRAM, 16GB RAM) hoping for a Qwen4 35B A3B-class release that fits their hardware. Until then, users rely on Gemma 26B QAT, which reportedly runs at about 26 tokens/second on such setups.
A LocalLLaMA thread captures low-VRAM hobbyists (8GB VRAM, 16GB RAM) hoping for a Qwen4 35B A3B-class release that fits their hardware. Until then, users rely on Gemma 26B QAT, which reportedly runs at about 26 tokens/second on such setups.
143 engagement1 sourceredditSat, Oct 3, 2026, 02:16 PM
Social·Community
Redditors weigh Qwen3.8 Flash-Next vs GLM 5.3 vs DeepSeek 4 Flash for a 128GB VRAM build
A Reddit user building a 128GB VRAM rig asks which large open models to run locally: Qwen3.8 Flash-Next, GLM 5.3, or DeepSeek 4 Flash. The thread reflects a practitioner trade-off between quantization levels and model reception.
A Reddit user building a 128GB VRAM rig asks which large open models to run locally: Qwen3.8 Flash-Next, GLM 5.3, or DeepSeek 4 Flash. The thread reflects a practitioner trade-off between quantization levels and model reception.
77 engagement1 sourceredditThu, Sep 10, 2026, 12:50 PM
Social·Community
User seeks advice on local inference build with dual 3090s and NVLink for Qwen Coder Next
A Reddit user details their current local inference setup using a 7900 XTX for Qwen 3.6 35B-A3B and asks for advice on building a dual 3090 system with NVLink and vLLM to run Qwen Coder Next (38GB). The post highlights practical considerations for tensor parallelism and NVLink's role in inference vs. training.
A Reddit user details their current local inference setup using a 7900 XTX for Qwen 3.6 35B-A3B and asks for advice on building a dual 3090 system with NVLink and vLLM to run Qwen Coder Next (38GB). The post highlights practical considerations for tensor parallelism and NVLink's role in inference vs. training.
13 engagement1 sourceredditSat, Jul 25, 2026, 07:01 PM