Releases·Tool Release
Community member releases imatrix GGUF quants for Qwen3.8-Flash-Next, 20-30GB smaller than Unsloth builds
A community quantizer released mainline-compatible imatrix GGUF quants for Qwen3.8-Flash-Next on Hugging Face after several days of testing. The builds target the same quality band as popular Unsloth and AesSedai Q4 quants while saving roughly 20-30GB of disk and RAM, with perplexity competitive with both. This gives local inference users smaller files without a quality tradeoff.
39 engagement1 sourceSat, Aug 29, 2026, 03:02 PM
AI Overview
- Repo published at huggingface.co/agentionai/Qwen3.8-Flash-Next-AP-GGUF with perplexity numbers in the model card
- Savings of roughly 20-30GB versus Unsloth and AesSedai Q4 builds depending on the file compared
- Author recommends starting with the Q4 quants; Q3 and Q5 variants are planned
- Recipe is per-layer (excerpt truncated at 'per-la')
Related
Social·Community
Qwen 3.8 flash for a single spark
A community member released a low-bit quantization of the Qwen 3.8 Flash base model on Hugging Face. The quant reportedly retains 95% of bfloat16 accuracy without degradation at long contexts and reaches roughly 40 tokens/s, leaving headroom for 256k context and RoPE. Inference improvements are still in progress.
A community member released a low-bit quantization of the Qwen 3.8 Flash base model on Hugging Face. The quant reportedly retains 95% of bfloat16 accuracy without degradation at long contexts and reaches roughly 40 tokens/s, leaving headroom for 256k context and RoPE. Inference improvements are still in progress.
3 engagement1 sourceredditFri, Oct 2, 2026, 11:27 AM
Social·Community
Redditor plans €3,000+ budget CPU/GPU hybrid build to run Qwen3.8-Flash-Next at 4-bit locally
A Reddit user planning their first local inference PC asked for feedback on a hybrid CPU+GPU build centered on Qwen3.8-Flash-Next at 4-bit, after testing roughly 20 open models. Qwen3.8-Flash-Next with medium reasoning was the only model that completed their task without inventing config options when a harness forced doc lookups. The plan puts most model weights in DDR5 RAM with a used RTX 3090 handling the remainder, and the user questions whether the CPU, RAM capacity, and memory speed choices will work.
A Reddit user planning their first local inference PC asked for feedback on a hybrid CPU+GPU build centered on Qwen3.8-Flash-Next at 4-bit, after testing roughly 20 open models. Qwen3.8-Flash-Next with medium reasoning was the only model that completed their task without inventing config options when a harness forced doc lookups. The plan puts most model weights in DDR5 RAM with a used RTX 3090 handling the remainder, and the user questions whether the CPU, RAM capacity, and memory speed choices will work.
16 engagement1 sourceredditTue, Oct 6, 2026, 11:32 AM
Releases·Tool Release
akopytko releases Blackwell-native NVFP4 quant of Qwen3.8 27B, 50% faster than Q4
A new GGUF quant of Qwen3.8 27B uses Blackwell-native NVFP4 with a prefill-optimized design, achieving 50% faster prefill than a Q4 quant of the same memory footprint and 4-7% faster than other NVFP4 quants on an RTX 5090. It includes a quantized MTP draft head and is available on Hugging Face.
A new GGUF quant of Qwen3.8 27B uses Blackwell-native NVFP4 with a prefill-optimized design, achieving 50% faster prefill than a Q4 quant of the same memory footprint and 4-7% faster than other NVFP4 quants on an RTX 5090. It includes a quantized MTP draft head and is available on Hugging Face.
166 engagement1 sourceredditFri, Aug 21, 2026, 09:19 AM
Releases·Tool Release
Show HN: LoRA over GGUF – Train Qwen3.8-Flash-Next in 40G VRAM
A developer released a training method that applies LoRA directly over GGUF-quantized models, enabling local QLoRA-style fine-tuning of MoE models that bnb 4-bit does not support. Using GGUF support added in Transformers 5.18, they trained Qwen3.8-Flash-Next (125B-A6B + 51B engram) in 40 GiB VRAM without CPU offload, reaching 200 token/s on Strix Halo. The work opens a path for modifying open-weight MoE models on local hardware, complementing inference advances already achieved with GGUF.
A developer released a training method that applies LoRA directly over GGUF-quantized models, enabling local QLoRA-style fine-tuning of MoE models that bnb 4-bit does not support. Using GGUF support added in Transformers 5.18, they trained Qwen3.8-Flash-Next (125B-A6B + 51B engram) in 40 GiB VRAM without CPU offload, reaching 200 token/s on Strix Halo. The work opens a path for modifying open-weight MoE models on local hardware, complementing inference advances already achieved with GGUF.
1 engagement1 sourcehackernewsSat, Oct 10, 2026, 04:43 PM
Releases·Tool Release
ISTA-DASLab releases GSQ-RCO GGUFs for Qwen3.8-Flash-Next: near-baseline quality at 68-76GB, up to 3.4x prompt throughput
ISTA-DASLab published GSQ-RCO quantized GGUFs of Qwen3.8-Flash-Next on Hugging Face, shrinking the model from ~80-95GB to 68-76GB while preserving near baseline quality. The release targets practitioners running the model locally at reduced memory cost.
ISTA-DASLab published GSQ-RCO quantized GGUFs of Qwen3.8-Flash-Next on Hugging Face, shrinking the model from ~80-95GB to 68-76GB while preserving near baseline quality. The release targets practitioners running the model locally at reduced memory cost.
115 engagement1 sourceredditWed, Sep 16, 2026, 11:09 AM