Releases·Tool Release
stuntd: a local Jev-compatible server on Laya that learns from your own traffic (no API key needed)
A developer released stuntd, an Apache-2.0 local proxy for typed LLM decisions (choice, yes/no, score) that records traffic from a provider like the hosted Jev API, trains a small head on the frozen open-weights Laya encoder, and serves answers locally with fallback to the provider below a confidence threshold. It matters to practitioners because it cuts ~380 ms paid round trips down to ~22 ms local inference on closed-decision tasks, with self-promotion and demotion based on agreement rates.
33 engagement1 sourceWed, Sep 23, 2026, 02:13 AM
AI Overview
- stuntd is a local proxy recording typed LLM decisions (choice, yes/no, score), training a frozen-encoder head on Laya, and serving live only after reaching a target agreement with the provider
- Below the confidence threshold requests fall back to the provider; it demotes itself if agreement drops, and passes Anthropic, Responses, and Gemini traffic through untouched
- Benchmarks on RTX 5060 laptop: head answers in 22 ms CUDA / 60 ms CPU; Snake agent jumps from 0.7 zero-shot to 11.4 score after 30 teacher games and 138 s of training
- Support triage agreement with teacher rises from 69/34/66% zero-shot to 99/86/98% trained, answering 90% of questions locally at a 0.99 target; command gate hits 97% trained from 45% zero-shot
- Training caches frozen encoder outputs per site, so 24 epochs over 3000 rows take 2-6 minutes; each serve loads a 1.6 GB checkpoint
- Laya scores 38% zero-shot on banking77 (77 intents) vs Jev's 76% per dhruvmehra/jevbench; trained head reaches 100% over 12-label banking intents
- It only learns closed decisions, never free text: a risk rule phrased as "amount > X" scored 0.42 versus 0.94 phrased in words; repo is Apache-2.0
- Includes a 20-line PreToolUse hook that asks a local command_gate question before Claude Code runs shell commands
Related
Products·launched
Stuntd: LLM proxy that distills repeated small LLM decisions into small local models
Stuntd is a proxy that watches an app's traffic to an LLM, learns a small classifier head per repeated closed decision (routing, urgency, safety checks), and serves those decisions locally. It runs in shadow mode and takes over only when agreement with the teacher model meets a target threshold, cutting per-call provider costs. Below its confidence threshold requests fall back to the provider, and it demotes itself if it drifts.
Stuntd is a proxy that watches an app's traffic to an LLM, learns a small classifier head per repeated closed decision (routing, urgency, safety checks), and serves those decisions locally. It runs in shadow mode and takes over only when agreement with the teacher model meets a target threshold, cutting per-call provider costs. Below its confidence threshold requests fall back to the provider, and it demotes itself if it drifts.
2 engagement2 sourcesredditWed, Sep 23, 2026, 03:59 AM
Social·Communityfeatured
Jev docs review: decision-routing tool that picks between LLM outputs, not a Claude or ChatGPT replacement
A wave of developer posts examines Jev, TypeSafe's 'System 1' model that returns typed decisions with confidence scores instead of generating text. A production startup reports near-human computer-use latency, a docs review maps where it can and cannot replace Claude, and others replicate the mechanism with llama.cpp logprobs, a GitHub tool, and Laya, an Apache-2.0 open clone that hit 13k stars three days after Jev's early access. The debate centers on whether Jev has a moat beyond packaging a calibrated classifier pattern.
A wave of developer posts examines Jev, TypeSafe's 'System 1' model that returns typed decisions with confidence scores instead of generating text. A production startup reports near-human computer-use latency, a docs review maps where it can and cannot replace Claude, and others replicate the mechanism with llama.cpp logprobs, a GitHub tool, and Laya, an Apache-2.0 open clone that hit 13k stars three days after Jev's early access. The debate centers on whether Jev has a moat beyond packaging a calibrated classifier pattern.
Updated Wed, Sep 23, 2026, 06:33 PM — Open-source clone Laya (fine-tuned ModernBERT, Apache 2.0, 13k stars) and llama.cpp logprob recipes replicate Jev's mech
452900 engagement14 sourcesreddit, rssSun, Sep 20, 2026, 12:49 AM
Releases·Tool Release
stuntd 0.1.2: local heads for multi-field decisions, and why one weak field decides how often you skip the model
The stuntd developer released v0.1.2 of their proxy that learns an LLM's typed decisions and answers confident cases with small local heads (~20ms GPU, ~60ms CPU). The update adds multi-field decisions handled all-or-nothing, automatic shadow retraining, Anthropic Messages support, and a lazy serving mode.
The stuntd developer released v0.1.2 of their proxy that learns an LLM's typed decisions and answers confident cases with small local heads (~20ms GPU, ~60ms CPU). The update adds multi-field decisions handled all-or-nothing, automatic shadow retraining, Anthropic Messages support, and a lazy serving mode.
0 engagement1 sourceredditWed, Sep 30, 2026, 04:06 PM
Releases·Model Releasefeatured
Developer releases Laya, a 421M-parameter open-source decision model trained with RLCD, with HF space, benchmarks, and repo
Convai Innovations released Laya, an Apache 2.0 non-autoregressive decision model built by Nandakishor M as an open alternative to TypeSafe's closed Jev API. It answers typed questions (choice, score, noul) in a single ~35 ms forward pass, with checkpoints on ModernBERT-large and mmBERT-base, and a community C++ implementation followed within days.
Convai Innovations released Laya, an Apache 2.0 non-autoregressive decision model built by Nandakishor M as an open alternative to TypeSafe's closed Jev API. It answers typed questions (choice, score, noul) in a single ~35 ms forward pass, with checkpoints on ModernBERT-large and mmBERT-base, and a community C++ implementation followed within days.
Updated Sun, Sep 27, 2026, 11:55 PM — Community member lkarlslund released laya.cpp, an MIT-licensed C++ inference implementation with custom CUDA kernels and
211860 engagement4 sourcesredditFri, Sep 18, 2026, 06:25 AM
Products·launched
shad0w: caching proxy that learns a local LLM's classification answers and serves repeats in microseconds
shad0w is a proxy that sits in front of any OpenAI-compatible local LLM server, logs classification answers, and after ~1,000 logged answers compiles a small lookup table that serves repeated classifications on CPU in microseconds. It only answers where it matches the model 95%+ on held-out data, routing everything else to the model. It targets developers running local models for classification who want to eliminate repeat latency.
shad0w is a proxy that sits in front of any OpenAI-compatible local LLM server, logs classification answers, and after ~1,000 logged answers compiles a small lookup table that serves repeated classifications on CPU in microseconds. It only answers where it matches the model 95%+ on held-out data, routing everything else to the model. It targets developers running local models for classification who want to eliminate repeat latency.
1 engagement1 sourceredditMon, Oct 12, 2026, 07:33 AM