💾 Smaller KV cache. Bigger savings. Compared with the previous generation, V4.1-Flash’s KV cache needs just: 🔹 1/4 the HBM 🔹 1/8 the SSD storage Cache-hit charges often account for a large share of a

Developers released OpenLake, an open-source storage engine that offloads LLM KV caches from GPU memory to a shared tier of RAM and NVMe. It addresses the problem of KV caches exceeding GPU memory—e.g., a 256K-token conversation on Gemma 4 31B produces ~43GB of KV state—and eliminates redundant recomputation across a cluster. The approach reduces long-context inference costs by 50%.

OpenLake, an open-source storage engine for KV cache offloading, released ExANS, a lossless compression method that runs on GPU at 622 GB/s on H100. It compresses only the exponent stream of BF16 tensors, exploiting low entropy in real-world KV blocks to reduce PCIe/NIC bandwidth bottlenecks.

DeepSeek posts v4.1-Flash tech report on KV cache compression

Developments·Hardware

FlashAccel paper proposes High-Bandwidth Flash (HBF) for high-throughput LLM inference

An arXiv paper (2607.10186) presents FlashAccel, an approach using High-Bandwidth Flash (HBF) for high-throughput LLM inference. HBF reportedly offers 8x-16x more capacity than HBM at the same cost, with bandwidth up to 3 TB/s. The posts are discussion of the paper, not a vendor product launch.

An arXiv paper (2607.10186) presents FlashAccel, an approach using High-Bandwidth Flash (HBF) for high-throughput LLM inference. HBF reportedly offers 8x-16x more capacity than HBM at the same cost, with bandwidth up to 3 TB/s. The posts are discussion of the paper, not a vendor product launch.

A community analysis compares two KV cache compression methods, OSCAR and TurboQuant, concluding that OSCAR achieves 2.28 effective bits per element versus TurboQuant's higher effective bits due to reliance on a 1-bit residual corrector. OSCAR's hybrid three-segment topology enables ultra-lean compression.