Base Compute released BaseQuant V1, its first quantized versions of Qwen3.8-27B, on Hugging Face. The quants aim to preserve more of the original model at comparable file sizes, claiming lower KL divergence from BF16 than similarly sized Unsloth quants. Practitioners running local GGUF models get a new option for quality-per-gigabyte tradeoffs.
Unsloth releases Dynamic v3.1 quantizations for Qwen3.8-27B
Unsloth releases Dynamic v3.1 quantizations for Qwen3.8-27B
Two independent teams released aggressively quantized versions of Qwen3.8-27B within two days. One ships GGUF checkpoints at 2.5-3.0 bits per weight using two new methods, GSQ and RCO, with both the search and quantizer learned. The other releases an NVFP4 W4A4 checkpoint trained via quantization-aware distillation with the QUASAR QAT algorithm, deployable on vLLM with NVIDIA Blackwell GPUs. Both matter to practitioners running large models at low precision locally or on modern accelerators.
Two independent teams released aggressively quantized versions of Qwen3.8-27B within two days. One ships GGUF checkpoints at 2.5-3.0 bits per weight using two new methods, GSQ and RCO, with both the search and quantizer learned. The other releases an NVFP4 W4A4 checkpoint trained via quantization-aware distillation with the QUASAR QAT algorithm, deployable on vLLM with NVIDIA Blackwell GPUs. Both matter to practitioners running large models at low precision locally or on modern accelerators.
A Reddit user reports a task-aware quantization (TAK) pipeline for Qwen3.8-27B that scores 82.81% on reasoning benchmarks, versus 83.59% for BF16 and 77.34% for a byte-matched Unsloth UD IQ2_S quant. The work was built over several months and packaged into a repeatable pipeline, with coding as the next target domain. The result matters to practitioners running large models locally, suggesting domain-aware quants can approach full-precision quality at a fraction of the size.
A Reddit user reports a task-aware quantization (TAK) pipeline for Qwen3.8-27B that scores 82.81% on reasoning benchmarks, versus 83.59% for BF16 and 77.34% for a byte-matched Unsloth UD IQ2_S quant. The work was built over several months and packaged into a repeatable pipeline, with coding as the next target domain. The result matters to practitioners running large models locally, suggesting domain-aware quants can approach full-precision quality at a fraction of the size.
ISTA Deep Algorithms and Systems Lab released GGUF quantizations of the 176.9B-parameter sparse MoE model Qwen3.8-Flash-Next using two new methods, GSQ and RCO. Four quantized builds run from 2.40 to 3.50 bpw, and a Coder build removes half the routed experts, yielding an effective ~1.89 bits per original parameter and a 29.6 GB resident footprint that fits a single 32 GB accelerator. Benchmarks show the 3.50 bpw build matching BF16 on every evaluated benchmark.
ISTA Deep Algorithms and Systems Lab released GGUF quantizations of the 176.9B-parameter sparse MoE model Qwen3.8-Flash-Next using two new methods, GSQ and RCO. Four quantized builds run from 2.40 to 3.50 bpw, and a Coder build removes half the routed experts, yielding an effective ~1.89 bits per original parameter and a 29.6 GB resident footprint that fits a single 32 GB accelerator. Benchmarks show the 3.50 bpw build matching BF16 on every evaluated benchmark.
A user benchmarked two ultra-low-bit GGUF quants of Qwen 3.8 27B on an RTX 3060 12GB, comparing ISTA-DASLab's GSQ-RCO-IQ3-XXS against ByteShape's IQ3-XXS. Despite ByteShape's advertised high BF16 similarity, it failed tasks the GSQ quant completed in one shot. The result challenges ByteShape's KL-divergence-based quality claims for low-bit quantization.
A user benchmarked two ultra-low-bit GGUF quants of Qwen 3.8 27B on an RTX 3060 12GB, comparing ISTA-DASLab's GSQ-RCO-IQ3-XXS against ByteShape's IQ3-XXS. Despite ByteShape's advertised high BF16 similarity, it failed tasks the GSQ quant completed in one shot. The result challenges ByteShape's KL-divergence-based quality claims for low-bit quantization.