Releases·Tool Release
OrcaRouter ships runtime refusal-direction ablation for Ternary Bonsai 2 27B without editing weights
The OrcaRouter team released OrcaBonsai-27B-Uncensored, a runtime behavioral-ablation method that uncensors the 27B Ternary Bonsai 2 model at approximately 1.72 bits per weight without modifying or re-quantizing its weights. Traditional abliteration requires saving dense full-precision matrices, which would break the ternary quantization-aware training; instead, the projection is applied at inference. The original model pack stays bit-identical, making the approach relevant to anyone serving aggressively quantized models on constrained hardware.
450 engagement1 sourceFri, Sep 18, 2026, 06:45 AM
AI Overview
- OrcaRouter applies refusal-direction ablation to the 27B, ~1.72 bits/weight Ternary Bonsai 2 model entirely at runtime, leaving weights bit-identical.
- Runtime approach avoids re-quantization that aggressive 2-bit ternary weights, trained via quantization-aware training, could not survive.
- The ablation uses 129 residual intervention sites with runtime-adjustable strength, targeting prism-ml/Ternary-Bonsai-2-27B-mlx-2bit on Apple Silicon/MLX.
Entities
Related
Social·Community
Bonsai's own whitepaper shows Ternary Bonsai 2 27B trails Qwen3.5-27B despite 98.2% retention claim
A Reddit post argues Bonsai's headline claim of 98.2% intelligence retained for Ternary Bonsai 2 27B is cherry-picked. The company's own whitepaper reports scores roughly three quarters of Qwen3.5-27B's on Terminal-Bench 2.1 and SWE-bench Verified, and a possible typo suggests the comparison baseline may actually be Qwen3.8.
A Reddit post argues Bonsai's headline claim of 98.2% intelligence retained for Ternary Bonsai 2 27B is cherry-picked. The company's own whitepaper reports scores roughly three quarters of Qwen3.5-27B's on Terminal-Bench 2.1 and SWE-bench Verified, and a possible typo suggests the comparison baseline may actually be Qwen3.8.
198 engagement1 sourceredditFri, Sep 18, 2026, 11:26 AM
Releases·Model Release
Prism ML releases Ternary Bonsai 2, a 27B ternary-weight model under 6GB that runs in-browser on WebGPU
Prism ML released Ternary Bonsai 2 on Hugging Face, a 27B model using ternary weights to shrink to under 6GB. Derived from Qwen3.8-27B with architecture unchanged, it is 9x smaller than FP16 while retaining 98.2% of its intelligence, and can run in-browser via WebGPU. This makes a large hybrid-attention model practical for fully local, client-side inference.
Prism ML released Ternary Bonsai 2 on Hugging Face, a 27B model using ternary weights to shrink to under 6GB. Derived from Qwen3.8-27B with architecture unchanged, it is 9x smaller than FP16 while retaining 98.2% of its intelligence, and can run in-browser via WebGPU. This makes a large hybrid-attention model practical for fully local, client-side inference.
1.9k engagement1 sourceredditThu, Sep 17, 2026, 09:05 PM
Developments·Paper
Arxiv paper proposes ternary multiplicative adaptation for transformer fine-tuning
A research paper introduces a method to directly fine-tune ternary transformers using low-rank Kronecker factorization. This bypasses dequantization requirements of existing low-bit LoRA approaches, maintaining model efficiency. The technique enables discrete weight updates like sign flips while preserving ternary format.
A research paper introduces a method to directly fine-tune ternary transformers using low-rank Kronecker factorization. This bypasses dequantization requirements of existing low-bit LoRA approaches, maintaining model efficiency. The technique enables discrete weight updates like sign flips while preserving ternary format.
0 engagement1 sourcearxivTue, Aug 25, 2026, 12:15 PM
Releases·Tool Release
KaozKit: sandboxed LLM agents in JS, fully on-device on Apple Silicon (MLX / Apple Intelligence / Ollama), MIT
Sebastien Burel released KaozKit, an MIT-licensed Swift library that runs JavaScript LLM agents inside Mac apps fully on-device. Agents execute in XS, a microcontroller-oriented JS engine, sandboxed so they see only host-provided functions. It supports MLX, Apple Intelligence's on-device model, Ollama, and LM Studio.
Sebastien Burel released KaozKit, an MIT-licensed Swift library that runs JavaScript LLM agents inside Mac apps fully on-device. Agents execute in XS, a microcontroller-oriented JS engine, sandboxed so they see only host-provided functions. It supports MLX, Apple Intelligence's on-device model, Ollama, and LM Studio.
1 engagement1 sourceredditThu, Sep 17, 2026, 08:36 AM
Developments·Paper
AWSRC: Seeded residual coding repairs INT4 LLM weights with 0.162 bits/weight
A new paper introduces Activation-Weighted Seeded Residual Coding (AWSRC), a compact codec that repairs errors from low-bit weight quantization in LLMs. By encoding residuals with seed-generated bases and prioritizing activation-sensitive weights, AWSRC adds only 0.162 bits per weight to an INT4 RTN backbone, closing 88.2% of the quality gap on Qwen2.5-3B-Instruct.
A new paper introduces Activation-Weighted Seeded Residual Coding (AWSRC), a compact codec that repairs errors from low-bit weight quantization in LLMs. By encoding residuals with seed-generated bases and prioritizing activation-sensitive weights, AWSRC adds only 0.162 bits per weight to an INT4 RTN backbone, closing 88.2% of the quality gap on Qwen2.5-3B-Instruct.
0 engagement1 sourcearxivMon, Aug 24, 2026, 11:49 AM