VeriLoop E2 GGUF Release: BF16 → IQ1_M, 50.11 GiB → 16.79 GiB with +0.319% PPL drift

Releases·Tool Release

Two new quantization releases target Qwen3.8-27B: GSQ-RCO GGUFs and QUASAR NVFP4

Two independent teams released aggressively quantized versions of Qwen3.8-27B within two days. One ships GGUF checkpoints at 2.5-3.0 bits per weight using two new methods, GSQ and RCO, with both the search and quantizer learned. The other releases an NVFP4 W4A4 checkpoint trained via quantization-aware distillation with the QUASAR QAT algorithm, deployable on vLLM with NVIDIA Blackwell GPUs. Both matter to practitioners running large models at low precision locally or on modern accelerators.

Two independent teams released aggressively quantized versions of Qwen3.8-27B within two days. One ships GGUF checkpoints at 2.5-3.0 bits per weight using two new methods, GSQ and RCO, with both the search and quantizer learned. The other releases an NVFP4 W4A4 checkpoint trained via quantization-aware distillation with the QUASAR QAT algorithm, deployable on vLLM with NVIDIA Blackwell GPUs. Both matter to practitioners running large models at low precision locally or on modern accelerators.

Social·Community

FP8/BF16 users seeing someone run IQ1_S

Community meme mocks FP8/BF16 users' reactions to IQ1_S quantization

Community meme mocks FP8/BF16 users' reactions to IQ1_S quantization

A Reddit user explains the practical difference between GGUF Q4_K_M and Q4_K_S quantizations: the Q number is nominal, not literal, and k-quants keep damage-sensitive tensors (attention, embeddings) at higher precision while compressing the rest harder. The M variant protects more sensitive tensors, which is why its file is larger.

A Reddit user reports LM Studio now accepts llama.cpp parameter overrides, previously requiring re-saving the GGUF to change load arguments. They suggest setting --yarn-attn-factor to 1.2 to reduce signal loss between layers, claiming it improves creativity-oriented outputs. The tip comes with a shareable JSON config for LM Studio's import feature and caveats that results vary by model.

Releases·Tool Release

Transformers now runs llama.cpp quants

Hugging Face announced that transformers can now load and run llama.cpp GGUF quantized checkpoints directly via from_pretrained, reusing ggml's Metal kernels through the kernels library to approach llama.cpp performance. Initial support targets Apple Silicon Macs, starting with the Qwen3.5 architecture. This bridges llama.cpp's local inference foundation with transformers' PyTorch model-definition ecosystem, enabling experimentation, evaluation, fine-tuning, and conversion validation on GGUF checkpoints.

Hugging Face announced that transformers can now load and run llama.cpp GGUF quantized checkpoints directly via from_pretrained, reusing ggml's Metal kernels through the kernels library to approach llama.cpp performance. Initial support targets Apple Silicon Macs, starting with the Qwen3.5 architecture. This bridges llama.cpp's local inference foundation with transformers' PyTorch model-definition ecosystem, enabling experimentation, evaluation, fine-tuning, and conversion validation on GGUF checkpoints.