Releases·Tool Release
Community fork beellama-kvarn reports up to 76% faster text generation with kvarn KV quants
A new fork of beellama, beellama-kvarn, optimizes kvarn KV quantization so text generation speed holds up at high context depths. The author reports up to 76% faster text generation than beellama's own kvarn implementation, with performance matching or beating llama.cpp at equivalent qx_x quants. This matters to practitioners running long-context local inference where KV quant usually degrades speed.
10 engagement1 sourceMon, Aug 31, 2026, 08:03 PM
AI Overview
- Fork at github.com/valujin/beellama-kvarn optimizes kvarn KV quants, which previously tanked text generation speed as context depth grew
- Author's testing shows up to 76% faster text generation than beellama's kvarn implementation
- At low context depth, kvarn quants perform similarly to llama.cpp at an equivalent qx_x quant
- At equivalent qx_x quants and high context depths, the fork matches or beats llama.cpp performance
- Author's testing capped at context length 99328; gains expected to grow at higher contexts
- GitHub (translated from Russian) reports results at context 163840
Related
Releases·Tool Release
ConvRot quantization method lands in llama-cpp-turboquant
The ConvRot quantization method, previously shown to improve accuracy over UDQ8, is now implemented in the llama-cpp-turboquant fork. Users report Q6_CR and Q5_CR variants offer slight quality gains over their base counterparts, with potential to recover quality lost in turbo4/3/2 quantizations.
The ConvRot quantization method, previously shown to improve accuracy over UDQ8, is now implemented in the llama-cpp-turboquant fork. Users report Q6_CR and Q5_CR variants offer slight quality gains over their base counterparts, with potential to recover quality lost in turbo4/3/2 quantizations.
74 engagement1 sourceredditMon, Aug 24, 2026, 01:16 AM
Releases·Tool Release
BeeLlama.cpp v0.4.0 adds KVarN, KV precision tail, and more KV cache quantization types
BeeLlama.cpp v0.4.0, a fork of llama.cpp, introduces KVarN, KV cache precision tail, and additional KV cache quantization types (q2_0–q3_1, q6_0, q6_1). The update removes DFlash and TurboQuant+TCQ as they are now upstream or underperformed. All claims are supported by benchmarks.
BeeLlama.cpp v0.4.0, a fork of llama.cpp, introduces KVarN, KV cache precision tail, and additional KV cache quantization types (q2_0–q3_1, q6_0, q6_1). The update removes DFlash and TurboQuant+TCQ as they are now upstream or underperformed. All claims are supported by benchmarks.
102 engagement2 sourcesredditSun, Jul 19, 2026, 06:06 PM
Releases·Tool Release
llama.cpp PR reports up to 169% faster quantized-KV decode at 118K context on Intel Battlemage from one SYCL kernel switch
A llama.cpp pull request (#26689) switches the SYCL FlashAttention dispatch for quantized KV caches from the VEC kernel to the TILE kernel, yielding dramatic decode speedups on Intel Battlemage GPUs at long context. The change is small but delivers up to 169% faster token generation for models like Qwen3.6-35B and Gemma 4 12B at 118K context.
A llama.cpp pull request (#26689) switches the SYCL FlashAttention dispatch for quantized KV caches from the VEC kernel to the TILE kernel, yielding dramatic decode speedups on Intel Battlemage GPUs at long context. The change is small but delivers up to 169% faster token generation for models like Qwen3.6-35B and Gemma 4 12B at 118K context.
125 engagement1 sourceredditFri, Aug 7, 2026, 05:09 PM
Releases·Tool Release
Llama.cpp fork adds adaptive speculation, boosting token generation up to 50%
A developer introduced adaptive speculation for Llama.cpp, a fork that automatically adjusts the number of suggested tokens based on content type, improving token generation by up to 50% over mainline. The feature targets dense models like Qwen3.8, where MTP and DFlash settings vary by content.
A developer introduced adaptive speculation for Llama.cpp, a fork that automatically adjusts the number of suggested tokens based on content type, improving token generation by up to 50% over mainline. The feature targets dense models like Qwen3.8, where MTP and DFlash settings vary by content.
141 engagement1 sourceredditTue, Aug 25, 2026, 11:35 AM