Cut token consumption by 88% with deterministic image routing (P50: 59ms) [P]

Releases·Tool Release

Local Context Router cuts vision-token waste by pre-classifying PDF pages

A developer built LocalContextRouter, a tool that runs locally to classify each PDF page as text, OCR-needed, or image before sending to a multimodal model. This avoids the wasteful practice of rendering every page to an image, which can cost 1,300–4,800 tokens per page versus 400–800 tokens for plain text. The tool never calls a model itself, leaving the final API call to the user's app.

A developer built LocalContextRouter, a tool that runs locally to classify each PDF page as text, OCR-needed, or image before sending to a multimodal model. This avoids the wasteful practice of rendering every page to an image, which can cost 1,300–4,800 tokens per page versus 400–800 tokens for plain text. The tool never calls a model itself, leaving the final API call to the user's app.

An arXiv paper (2026-09-09) argues that fixed vision token pruning strategies underperform because per-image requirements differ. The authors show that strategies ranked best by average benchmark accuracy lose on a significant fraction of individual samples, motivating routing different pruning strategies per input to cut MLLM inference costs.

E2S-Pruner is a new visual token pruning method for vision-language models that uses progressive two-stage evidence fusion, requiring no auxiliary model, trainable parameters, or fine-tuning. It addresses limitations of existing attention-based pruning methods by better handling evidential uncertainty and conflict across attention heads and layers.

An arXiv paper introduces VPRune, a training-free framework that prunes visual tokens before the LLM stage of large vision-language models to cut inference cost. It addresses three causes of degradation from aggressive pruning: text-guided selection bias, information loss from discarded tokens, and positional distortion from sequence compaction. Testing shows favorable accuracy-compression trade-offs and reduced end-to-end latency on edge devices.

A new training-free framework, RoRA, prunes visual tokens in multimodal LLMs by allocating them into roles—semantic core, complementary context, and fine-grained details—to preserve object-related regions. It reduces prefilling and KV-cache costs without retraining, addressing a key efficiency bottleneck in MLLMs.