The QUASAR team released quantization-aware distillation checkpoints for Qwen3.5-4B, Gemma 4 E4B/12B, and Muse Glimmer 30B in NVFP4 and Q4_0 GGUF formats. Each quantized model is distilled from its BF16 teacher on the teacher's own responses, achieving lower KL divergence than Google's, NVIDIA's, and Meta's official 4-bit releases. Checkpoints, code, a paper, and a project page are public on Hugging Face, GitHub, and arXiv.
Two independent teams released aggressively quantized versions of Qwen3.8-27B within two days. One ships GGUF checkpoints at 2.5-3.0 bits per weight using two new methods, GSQ and RCO, with both the search and quantizer learned. The other releases an NVFP4 W4A4 checkpoint trained via quantization-aware distillation with the QUASAR QAT algorithm, deployable on vLLM with NVIDIA Blackwell GPUs. Both matter to practitioners running large models at low precision locally or on modern accelerators.
Two independent teams released aggressively quantized versions of Qwen3.8-27B within two days. One ships GGUF checkpoints at 2.5-3.0 bits per weight using two new methods, GSQ and RCO, with both the search and quantizer learned. The other releases an NVFP4 W4A4 checkpoint trained via quantization-aware distillation with the QUASAR QAT algorithm, deployable on vLLM with NVIDIA Blackwell GPUs. Both matter to practitioners running large models at low precision locally or on modern accelerators.
A developer extended their MXFP4 vLLM build on the Radiance image with online NVFP4-to-MXFP4 conversion, letting NVFP4-quantized models use the MXFP4 fast path on AMD R9700 GPUs. Running Unsloth's Qwen3.8-27B NVFP4 quant, they measured 5,809 tok/s prefill and 276 tok/s decode via their BetterBench harness.
A developer extended their MXFP4 vLLM build on the Radiance image with online NVFP4-to-MXFP4 conversion, letting NVFP4-quantized models use the MXFP4 fast path on AMD R9700 GPUs. Running Unsloth's Qwen3.8-27B NVFP4 quant, they measured 5,809 tok/s prefill and 276 tok/s decode via their BetterBench harness.
A paper introduces Minima, applying NVFP4 W4A4 quantization to all 496 linear layers of Qwen3.8-27B, including the Gated DeltaNet recurrent blocks that community 4-bit quantizations had kept at higher precision. It challenges the intuition that recurrent linear-attention layers cannot tolerate 4-bit quantization.
A paper introduces Minima, applying NVFP4 W4A4 quantization to all 496 linear layers of Qwen3.8-27B, including the Gated DeltaNet recurrent blocks that community 4-bit quantizations had kept at higher precision. It challenges the intuition that recurrent linear-attention layers cannot tolerate 4-bit quantization.
A quantizer re-quantized Qwen3.5-9B with new techniques, claiming 31 wins, 3 ties, 0 losses across 34 size-matched comparisons against published GGUFs from Unsloth, bartowski, and others. The Q4_K_XL variant reportedly beats Unsloth's by 23% on KLD.
A quantizer re-quantized Qwen3.5-9B with new techniques, claiming 31 wins, 3 ties, 0 losses across 34 size-matched comparisons against published GGUFs from Unsloth, bartowski, and others. The Q4_K_XL variant reportedly beats Unsloth's by 23% on KLD.
A team shares low-bit quantization findings and releases QUASAR, a QAT method that keeps reconstructing quantized weights in a loss-aware way with no special inference path. On Qwen3.5-4B at 4 bits, good PTQ nearly matches QAT, but at 2 bits QUASAR beats the strongest QAT/PTQ baselines by 3-4 points on average across 8 tasks.
A team shares low-bit quantization findings and releases QUASAR, a QAT method that keeps reconstructing quantized weights in a loss-aware way with no special inference path. On Qwen3.5-4B at 4 bits, good PTQ nearly matches QAT, but at 2 bits QUASAR beats the strongest QAT/PTQ baselines by 3-4 points on average across 8 tasks.