Releases·Tool Release
Spanda: Sub-microsecond LLM epistemic uncertainty in Rust
The Spanda team released an open-source Rust engine and gateway for LLM epistemic uncertainty quantification and hallucination gating, with a Python SDK (pip install spnda). It replaces the semantic entropy NLI cross-encoder bottleneck (90ms GPU overhead with DeBERTa) with exact-match normalized entropy, achieving equal AUROC on GSM8K in 760 nanoseconds on CPU. The approach makes real-time production hallucination gating feasible without GPUs.
19 engagement1 sourceFri, Sep 11, 2026, 06:48 PM
AI Overview
- Adarshent open-sourced Spanda, a Rust engine gating LLM hallucinations via exact-match normalized entropy, hitting 760 ns CPU latency vs Semantic Entropy's 90 ms DeBERTa overhead
- Spanda matches Semantic Entropy's AUROC on GSM8K while using ~3 MB memory and 3.6 ms cold start versus Python proxies at 230 MB and 1.17 s
Related
Releases·Tool Release
Detecting hallucinations in local models without eating VRAM: What we learned testing 1.5B to 120B models
A team released Spanda (R_sc), a zero-dependency Python metric that detects hallucinations by sampling K responses at temperature 0.7 and measuring Shannon entropy over deterministically normalized strings on CPU — no NLI cross-encoder needed. Benchmarks across Qwen 1.5B, Mistral 7B, Qwen 27B, and a 120B frontier model on GSM8K and TriviaQA show it matches DeBERTa NLI performance for 7B+ models while running in 1.3 microseconds with zero GPU usage. Notably, the 120B model exhibited 'Confident Mode Collapse', repeating the identical wrong answer across all runs with zero entropy, defeating even NLI-based detection.
A team released Spanda (R_sc), a zero-dependency Python metric that detects hallucinations by sampling K responses at temperature 0.7 and measuring Shannon entropy over deterministically normalized strings on CPU — no NLI cross-encoder needed. Benchmarks across Qwen 1.5B, Mistral 7B, Qwen 27B, and a 120B frontier model on GSM8K and TriviaQA show it matches DeBERTa NLI performance for 7B+ models while running in 1.3 microseconds with zero GPU usage. Notably, the 120B model exhibited 'Confident Mode Collapse', repeating the identical wrong answer across all runs with zero entropy, defeating even NLI-based detection.
28 engagement1 sourceredditFri, Oct 2, 2026, 05:55 AM
Products·launched
Spanda: open-source CPU-based LLM hallucination detector
Spanda is an open-source, pure-Python hallucination detector that samples K LLM outputs and computes a normalized lexical consensus ratio, avoiding a second neural network. It is a lightweight, zero-dependency alternative to Semantic Entropy's DeBERTa-based clustering, letting developers — especially those running local models via Ollama — detect hallucinations without GPUs or extra VRAM.
Spanda is an open-source, pure-Python hallucination detector that samples K LLM outputs and computes a normalized lexical consensus ratio, avoiding a second neural network. It is a lightweight, zero-dependency alternative to Semantic Entropy's DeBERTa-based clustering, letting developers — especially those running local models via Ollama — detect hallucinations without GPUs or extra VRAM.
83 engagement2 sourcesredditSat, Sep 5, 2026, 06:56 AM
Releases·Tool Release
Truespar open-sources Paddock, a Rust/C++ inference engine with custom CUDA kernels, under MIT/Apache-2.0
The developers of Paddock, a Rust/C++ inference engine with its own CUDA kernels, released the code as open source on schedule, fulfilling an August commitment to ship in September. The public repo is their internal repo, including kernels, and the engine already serves roughly 300B tokens per year in production at their company. Practitioners get a single binary exposing OpenAI- and Anthropic-style APIs that loads GGUF and safetensors weights.
The developers of Paddock, a Rust/C++ inference engine with its own CUDA kernels, released the code as open source on schedule, fulfilling an August commitment to ship in September. The public repo is their internal repo, including kernels, and the engine already serves roughly 300B tokens per year in production at their company. Practitioners get a single binary exposing OpenAI- and Anthropic-style APIs that loads GGUF and safetensors weights.
92 engagement1 sourceredditFri, Sep 4, 2026, 09:15 AM
Developments·Paper
Kernel Token Contradiction: a Fast and Principled Approach for LLM Claim Uncertainty Quantification
A new paper introduces Kernel Token Contradiction (KTC), a lightweight method for claim-level uncertainty quantification in LLMs. KTC uses a positive semi-definite kernel integrating the model's conditional distribution and a token contradiction score, then applies Von Neumann entropy to measure uncertainty. It targets realistic white-box conditions.
A new paper introduces Kernel Token Contradiction (KTC), a lightweight method for claim-level uncertainty quantification in LLMs. KTC uses a positive semi-definite kernel integrating the model's conditional distribution and a token contradiction score, then applies Von Neumann entropy to measure uncertainty. It targets realistic white-box conditions.
0 engagement1 sourcearxivSun, Aug 23, 2026, 04:55 PM
Products·launchedfeatured
Rust+CUDA LLM inference engine for RTX 5090, optimized with NVFP4, MoE, and speculative decoding
A from-scratch LLM inference engine written in Rust and CUDA, specifically tuned for the RTX 5090 Laptop GPU (Blackwell sm_120a). It supports NVFP4 quantization, mixture-of-experts (MoE), and multi-token prediction (MTP) speculative decoding, achieving up to 1.6x speedup over llama.cpp on target models. Designed for developers who need maximum inference performance on a single high-end laptop GPU.
A from-scratch LLM inference engine written in Rust and CUDA, specifically tuned for the RTX 5090 Laptop GPU (Blackwell sm_120a). It supports NVFP4 quantization, mixture-of-experts (MoE), and multi-token prediction (MTP) speculative decoding, achieving up to 1.6x speedup over llama.cpp on target models. Designed for developers who need maximum inference performance on a single high-end laptop GPU.
260 engagement1 sourcegithubSun, Jul 5, 2026, 08:45 PM