Releases·Tool Release
llama.cpp fork deduplicates byte-identical messages in agent loops, cutting prompt tokens (e.g. 18,092 to 6,820)
A llama.cpp fork adds an opt-in pass to llama-server's chat parser that replaces byte-identical repeated messages in agent loops with one-line references, keeping the first copy intact. It is stateless and deterministic so prompt caches keep hitting, and it reports dedup stats in response timings. For agent workloads that repeatedly re-read files or re-send tool outputs, it cuts prompt sizes substantially.
10 engagement1 sourceSat, Sep 26, 2026, 02:02 PM
AI Overview
- Fork adds a llama-server chat-parser pass replacing later byte-identical same-role messages with one-line references; first copy stays in full
- Off by default; stateless and deterministic so prompt cache hits persist; enabled via --message-dedup with min-bytes and roles flags
- Eval of 15 synthetic agentic scenarios shows passing behavior unchanged with dedup on; prompts drop 18,092→6,820 and 108,197→71,697 tokens
- Response timings report dedup_n, dedup_bytes_saved, and dedup_tokens_saved_est; catches only exact repeats; wall-clock speedup not benchmarked
Related
Products·launched
Lossless context compressor for coding agents with prompt cache awareness
A lossless context compression tool for coding agents that preserves tool output (file contents, JSON, stack traces, diffs) exactly. It sits between the agent and the LLM, rewriting only blocks that can be proven exactly reconstructible, and attaches a BLAKE3 certificate for verification. Solves the problem of token waste in agent loops without losing critical information.
A lossless context compression tool for coding agents that preserves tool output (file contents, JSON, stack traces, diffs) exactly. It sits between the agent and the LLM, rewriting only blocks that can be proven exactly reconstructible, and attaches a BLAKE3 certificate for verification. Solves the problem of token waste in agent loops without losing critical information.
8 engagement1 sourceredditTue, Jul 21, 2026, 04:25 AM
Releases·Tool Release
I measured what happens when a coding agent reads the same file repeatedly
A developer released sqz, a local content-addressed cache for Claude Code sessions that replaces exact repeated file content with tiny references. The author observed agents rereading unchanged ~2,000-token files multiple times, paying full token cost each time. Across 3,003 compressions it saved 178,442 tokens with a 24.7% average reduction and no extra LLM call.
A developer released sqz, a local content-addressed cache for Claude Code sessions that replaces exact repeated file content with tiny references. The author observed agents rereading unchanged ~2,000-token files multiple times, paying full token cost each time. Across 3,003 compressions it saved 178,442 tokens with a 24.7% average reduction and no extra LLM call.
8 engagement1 sourceredditSun, Sep 20, 2026, 06:37 PM
Products·launched
Contextray: CLI tool to find and remove duplicate LLM context tokens
Contextray is a CLI tool that analyzes LLM context windows to identify exact duplicate chunks, such as those caused by agent loops, tool outputs, or framework bugs. It safely removes duplicates without breaking JSON or code, showing where tokens are wasted. It helps developers debugging multi-agent flows reduce token usage and costs.
Contextray is a CLI tool that analyzes LLM context windows to identify exact duplicate chunks, such as those caused by agent loops, tool outputs, or framework bugs. It safely removes duplicates without breaking JSON or code, showing where tokens are wasted. It helps developers debugging multi-agent flows reduce token usage and costs.
2 engagement1 sourceredditTue, Aug 11, 2026, 03:38 PM
Releases·Tool Release
Prompt-pruning layer for long-running LLM conversations released on GitHub
A deterministic, zero-dependency prompt-pruning layer for long-running LLM conversations has been released on GitHub. It expires stale tool state, collapses duplicate context, and provably never drops a fact a later turn still depends on.
A deterministic, zero-dependency prompt-pruning layer for long-running LLM conversations has been released on GitHub. It expires stale tool state, collapses duplicate context, and provably never drops a fact a later turn still depends on.
28 engagement1 sourcegithubWed, Jul 8, 2026, 02:02 AM
Releases·Tool Release
Developer releases nopus, a tool that detects and rewrites complex coding-agent prose
A developer released nopus, an open-source tool that deterministically detects unnecessarily complex prose in coding-agent responses and automatically requests a clearer rewrite from the same agent. It aims to reduce cognitive load when reading agent output, particularly from recent LLMs known for verbose language.
A developer released nopus, an open-source tool that deterministically detects unnecessarily complex prose in coding-agent responses and automatically requests a clearer rewrite from the same agent. It aims to reduce cognitive load when reading agent output, particularly from recent LLMs known for verbose language.
11 engagement1 sourceredditSat, Aug 15, 2026, 07:41 PM