Releases·Tool Release
I built a single C99 inference engine that runs both BitNet (ternary) and regular GGUF models - no Python, no CUDA
A developer released Project Zero, an inference engine written from scratch in pure C99 that runs both 1.58-bit ternary BitNet models and standard dense GGUF models as a single binary with no Python or CUDA. On a Xeon it reports 36 tok/s on BitNet b1.58-2B-4T, about 1.8x faster than bitnet.cpp. It ships with an OpenAI-compatible API and offers a pre-built Linux binary, though some quantization paths still lag llama.cpp.
6 engagement1 sourceMon, Sep 14, 2026, 05:13 PM
AI Overview
- Project Zero, written from scratch in pure C99 with only GCC and make, hits 36 tok/s on BitNet b1.58-2B-4T on a Xeon, 1.8x faster than bitnet.cpp
- Runs SmolLM2 F16 at ~100 tok/s on an i5-11300H, though llama.cpp beats it by ~7% on that hardware path
- Q4_K path lags llama.cpp badly: 1.9 tok/s vs 13.7 tok/s on DeepSeek, while ternary and dense F16 paths are solid
- Ships an OpenAI-compatible API (/v1/chat/completions) with SSE streaming and a pre-built x86 Linux binary on GitHub
Related
Releases·Tool Release
Project Zero: from-scratch C99 BitNet inference engine beats bitnet.cpp 1.8× on Xeon, seeks testers
A developer released Project Zero, a pure C99 CPU-only inference engine for BitNet b1.58-2B-4T and Qwen Bonsai-27B, achieving 36 tok/s on a Xeon — 1.8× faster than bitnet.cpp. The single-binary, zero-dependency tool is open-source and seeking community benchmarks.
A developer released Project Zero, a pure C99 CPU-only inference engine for BitNet b1.58-2B-4T and Qwen Bonsai-27B, achieving 36 tok/s on a Xeon — 1.8× faster than bitnet.cpp. The single-binary, zero-dependency tool is open-source and seeking community benchmarks.
69 engagement1 sourceredditWed, Jul 22, 2026, 07:02 PM
Releases·Tool Release
Developer builds zero-dependency C99 inference engine for BitNet 1.58-bit, hits 36 tok/s on Xeon
A developer created a CPU-first inference engine in pure C99 with no Python, CUDA, or BLAS dependencies, targeting 1.58-bit ternary models. The engine achieves 36.25 tok/s on BitNet b1.58-2B-4T using 4 threads on an Intel Xeon, leveraging custom AVX2/AVX-512 routines with VNNI instructions.
A developer created a CPU-first inference engine in pure C99 with no Python, CUDA, or BLAS dependencies, targeting 1.58-bit ternary models. The engine achieves 36.25 tok/s on BitNet b1.58-2B-4T using 4 threads on an Intel Xeon, leveraging custom AVX2/AVX-512 routines with VNNI instructions.
1 engagement1 sourceredditSat, Aug 8, 2026, 05:09 PM
Social·Community
Developer finds 29x matmul kernel gain yields only 6-10% end-to-end due to DRAM bottleneck in BitNet CPU inference engine
A developer building a C99 inference engine for BitNet's ternary models on CPU achieved a 29x speedup in a matmul kernel using AVX-512BW instructions, but end-to-end improvement is only 6-10% because DRAM bandwidth is the bottleneck. The kernel is not yet integrated into the full engine.
A developer building a C99 inference engine for BitNet's ternary models on CPU achieved a 29x speedup in a matmul kernel using AVX-512BW instructions, but end-to-end improvement is only 6-10% because DRAM bandwidth is the bottleneck. The kernel is not yet integrated into the full engine.
50 engagement1 sourceredditFri, Jul 24, 2026, 05:18 PM
Releases·Tool Release
Show HN: PicoLM v1.0-rc1 — C99 LLM inference engine for llama-2, GPT-2, Qwen, Gemma-3n
A developer released PicoLM v1.0-rc1, an LLM inference engine written in C99, on GitHub and announced it on Hacker News. It supports llama-2, GPT-2, Qwen 3.6/3.8 with MoE, and Gemma-3n models, with heavy emphasis on CPU SIMD acceleration and cross-platform portability. For practitioners, it offers a lightweight inference option spanning platforms from DOS to modern OSes.
A developer released PicoLM v1.0-rc1, an LLM inference engine written in C99, on GitHub and announced it on Hacker News. It supports llama-2, GPT-2, Qwen 3.6/3.8 with MoE, and Gemma-3n models, with heavy emphasis on CPU SIMD acceleration and cross-platform portability. For practitioners, it offers a lightweight inference option spanning platforms from DOS to modern OSes.
1 engagement1 sourcehackernewsThu, Sep 3, 2026, 08:19 AM
Releases·Tool Release
Truespar open-sources Paddock, a Rust/C++ inference engine with custom CUDA kernels, under MIT/Apache-2.0
The developers of Paddock, a Rust/C++ inference engine with its own CUDA kernels, released the code as open source on schedule, fulfilling an August commitment to ship in September. The public repo is their internal repo, including kernels, and the engine already serves roughly 300B tokens per year in production at their company. Practitioners get a single binary exposing OpenAI- and Anthropic-style APIs that loads GGUF and safetensors weights.
The developers of Paddock, a Rust/C++ inference engine with its own CUDA kernels, released the code as open source on schedule, fulfilling an August commitment to ship in September. The public repo is their internal repo, including kernels, and the engine already serves roughly 300B tokens per year in production at their company. Practitioners get a single binary exposing OpenAI- and Anthropic-style APIs that loads GGUF and safetensors weights.
92 engagement1 sourceredditFri, Sep 4, 2026, 09:15 AM