Hugging Face open-sources WebGPU kernel collection for fully local browser AI, plans upstreaming to Transformers.js and ONNX Runtime Web

Releases·Tool Release

Tritium: Open Source Ternary LLM Engine in Rust/CUDA for Consumer GPUs

Tritium, an open-source ternary LLM engine written in Rust and CUDA, has been released under Apache 2.0. It enables quantization, serving, and training of models on consumer GPUs, potentially lowering hardware barriers for LLM development.

Tritium, an open-source ternary LLM engine written in Rust and CUDA, has been released under Apache 2.0. It enables quantization, serving, and training of models on consumer GPUs, potentially lowering hardware barriers for LLM development.

A WebGPU-accelerated demo that runs a 1-bit quantized LLM entirely in the browser. It enables local, private text generation without server calls, targeting developers and researchers exploring edge inference.

Social·Community

Reddit megathread catalogs open source LLM optimization projects and MoE edge-inference research papers

A subreddit megathread compiles community-gathered open source projects focused on inference optimization, efficiency, and accessibility for open LLMs. The initial list includes Dwarfstar, an inference engine optimization proposal, and Colibri, a memory-hierarchy tool for MoE inference.

A subreddit megathread compiles community-gathered open source projects focused on inference optimization, efficiency, and accessibility for open LLMs. The initial list includes Dwarfstar, an inference engine optimization proposal, and Colibri, a memory-hierarchy tool for MoE inference.

A developer launched a browser-based runtime that executes PyTorch-style code via WebGPU, with no installation, server, or account. The TypeScript-based tool supports both model training and inference and was built to let students run torch code in classes without setup.

TinyDetect detects AI-generated text and images entirely client-side in the browser, so nothing users check (student essays, applications, photos) gets uploaded to a server. It quantizes two open detection models down to small sizes and runs them with ONNX Runtime Web. It is free with no sign-up.