Releases·Tool Release
an open-source tensor format that's 1.72x faster than safetensors for layer GEMV and loads 873M params in 15ms
A developer released HK, an open-source neural tensor framework and binary container format (.hk) positioned as a faster, hardware-aligned alternative to SafeTensors and GGUF. Written in Zig, it uses dual-mode weight reconstruction (4-bit lookup-table quantization plus a residual stream) to offer both compact edge-inference weights and on-demand exact precision. Benchmarks on Qwen3.5-0.8B show 1.72x faster layer GEMV and 346x faster warm autoregressive layer retrieval, and the project ships SDKs in seven languages under Apache 2.0.
3 engagement1 sourceMon, Sep 14, 2026, 11:01 AM
AI Overview
- HK (.hk) is a Zig-written, Apache 2.0 tensor format using dual-mode weight reconstruction: 4-bit quantization (~6.4x compression) plus an on-demand residual stream for >0.99999 cosine-similarity exact reconstruction.
- HK benchmarks on Qwen3.5-0.8B (873M bf16 params, 488 tensors): warm layer retrieval 80 ns vs 28.84 µs (346x), cold retrieval 136 µs vs 506 µs (3.72x), full load 15.88 ms via zero-copy mmap.
- HK supports 137+ architectures, lossless 2:4 structured sparsity (1.88x storage reduction), GGUF↔HK↔SafeTensors transcoding, NF4/Q4_K/Q8_K/MXFP4/BitNet quantization, and CLI tools at 2.8 MB idle RAM.
- HK ships SDKs in 7 languages (Python, Rust, TypeScript, Go, C#/.NET, Java/Android, C/C++) via packages hknt and hkntf.
Related
Releases·Model Release
GLM 5.2 compressed 86% to 250 GB via layer-wise quantization, per Unsloth's Daniel Han
Unsloth's Daniel Han claims GLM 5.2 can be compressed from 1.5 TB to 250 GB (86% smaller) without proportional quality loss, using layer-wise quantization informed by the super weights paper. The approach exploits unequal layer importance and near-zero parameters in models trained on 30 trillion tokens.
Unsloth's Daniel Han claims GLM 5.2 can be compressed from 1.5 TB to 250 GB (86% smaller) without proportional quality loss, using layer-wise quantization informed by the super weights paper. The approach exploits unequal layer importance and near-zero parameters in models trained on 30 trillion tokens.
40 engagement1 sourceyoutubeFri, Aug 7, 2026, 01:00 AM
Developments·Paper
Paper proposes HBQ, hierarchical block quantization for hardware-efficient accurate LLM inference
An arXiv paper introduces HBQ (Hierarchical Scaling Block Quantization), a design method for block quantization in LLM inference. Block quantization quantizes both weights and activations, enabling low-precision computation on a unified datapath with higher hardware efficiency than scalar weight-only quantization. The paper explores the underexplored design space of bit-width, block size, scaling, and numeric formats via design space exploration with hardware and benchmark results.
An arXiv paper introduces HBQ (Hierarchical Scaling Block Quantization), a design method for block quantization in LLM inference. Block quantization quantizes both weights and activations, enabling low-precision computation on a unified datapath with higher hardware efficiency than scalar weight-only quantization. The paper explores the underexplored design space of bit-width, block size, scaling, and numeric formats via design space exploration with hardware and benchmark results.
0 engagement1 sourcearxivMon, Aug 31, 2026, 10:39 PM
Developments·Paper
WarpQuant: INT3 PTQ for Qwen3.8-27B with Hadamard rotation and Output-Fisher recovery
WarpQuant is a new post-training quantization method that separates the compression domain from the recovery domain, achieving 3.6165 text bpw on Qwen3.8-27B. The release provides loadable safetensors with quantized values in BF16, though it is not yet Ollama-ready GGUF.
WarpQuant is a new post-training quantization method that separates the compression domain from the recovery domain, achieving 3.6165 text bpw on Qwen3.8-27B. The release provides loadable safetensors with quantized values in BF16, though it is not yet Ollama-ready GGUF.
1 engagement1 sourceredditSun, Aug 16, 2026, 01:58 AM
Social·Community
~22% less weight VRAM, lossless: base-3 packing for ternary GGUFs
A developer shared Q2_B3 ("B3S"), a denser GGUF packing scheme for ternary models like BitNet-b1.58 and Ternary-Bonsai. Because ternary weights take only -1, 0, +1 times a block scale, packing them in base 3 yields lossless 1.75 bits per weight instead of 2, cutting weight memory roughly 22%. For local-inference practitioners running ternary LLMs, this shrinks VRAM footprints without loss.
A developer shared Q2_B3 ("B3S"), a denser GGUF packing scheme for ternary models like BitNet-b1.58 and Ternary-Bonsai. Because ternary weights take only -1, 0, +1 times a block scale, packing them in base 3 yields lossless 1.75 bits per weight instead of 2, cutting weight memory roughly 22%. For local-inference practitioners running ternary LLMs, this shrinks VRAM footprints without loss.
38 engagement1 sourceredditFri, Sep 4, 2026, 07:04 PM
Developments·Paper
NVIDIA researchers publish H-Scale, a Hessian-guided method for NVFP4 quantization scale selection
An arXiv paper introduces H-Scale, a post-training quantization method that refines per-group scaling factors for NVFP4 sub-byte LLM inference on NVIDIA Blackwell hardware. It targets the underexplored scale-selection step rather than quantized weight values, improving accuracy of 4-bit inference.
An arXiv paper introduces H-Scale, a post-training quantization method that refines per-group scaling factors for NVFP4 sub-byte LLM inference on NVIDIA Blackwell hardware. It targets the underexplored scale-selection step rather than quantized weight values, improving accuracy of 4-bit inference.
0 engagement1 sourcearxivFri, Aug 28, 2026, 09:22 AM