Releases·Tool Release
Show HN: PicoLM v1.0-rc1 — C99 LLM inference engine for llama-2, GPT-2, Qwen, Gemma-3n
A developer released PicoLM v1.0-rc1, an LLM inference engine written in C99, on GitHub and announced it on Hacker News. It supports llama-2, GPT-2, Qwen 3.6/3.8 with MoE, and Gemma-3n models, with heavy emphasis on CPU SIMD acceleration and cross-platform portability. For practitioners, it offers a lightweight inference option spanning platforms from DOS to modern OSes.
1 engagement1 sourceThu, Sep 3, 2026, 08:19 AM
AI Overview
- PicoLM v1.0-rc1, a C99 LLM inference engine, supports llama-2, GPT-2, Qwen 3.6/3.8 (incl. MoE), and Gemma-3n models
- Runs on platforms from DOS to OS X 10.4 to modern systems, with CPU SIMD acceleration and CUDA/HIP plus accelerated IMMA kernels
- mmap()s GGUF files instead of preloading, using less RAM than llama.cpp; includes a LE/BE GGUF-to-FUGG endian-conversion utility
- Ships an OpenAI/llama.cpp-compatible HTTP server and an optional live VNC visualization of per-layer activation heatmaps
- Developer states prompt processing speed still needs work while text generation is already fast; v1.0-rc1 marks the first release candidate
Related
Releases·Tool Release
I built a single C99 inference engine that runs both BitNet (ternary) and regular GGUF models - no Python, no CUDA
A developer released Project Zero, an inference engine written from scratch in pure C99 that runs both 1.58-bit ternary BitNet models and standard dense GGUF models as a single binary with no Python or CUDA. On a Xeon it reports 36 tok/s on BitNet b1.58-2B-4T, about 1.8x faster than bitnet.cpp. It ships with an OpenAI-compatible API and offers a pre-built Linux binary, though some quantization paths still lag llama.cpp.
A developer released Project Zero, an inference engine written from scratch in pure C99 that runs both 1.58-bit ternary BitNet models and standard dense GGUF models as a single binary with no Python or CUDA. On a Xeon it reports 36 tok/s on BitNet b1.58-2B-4T, about 1.8x faster than bitnet.cpp. It ships with an OpenAI-compatible API and offers a pre-built Linux binary, though some quantization paths still lag llama.cpp.
6 engagement1 sourceredditMon, Sep 14, 2026, 05:13 PM
Releases·Tool Release
Truespar open-sources Paddock, a Rust/C++ inference engine with custom CUDA kernels, under MIT/Apache-2.0
The developers of Paddock, a Rust/C++ inference engine with its own CUDA kernels, released the code as open source on schedule, fulfilling an August commitment to ship in September. The public repo is their internal repo, including kernels, and the engine already serves roughly 300B tokens per year in production at their company. Practitioners get a single binary exposing OpenAI- and Anthropic-style APIs that loads GGUF and safetensors weights.
The developers of Paddock, a Rust/C++ inference engine with its own CUDA kernels, released the code as open source on schedule, fulfilling an August commitment to ship in September. The public repo is their internal repo, including kernels, and the engine already serves roughly 300B tokens per year in production at their company. Practitioners get a single binary exposing OpenAI- and Anthropic-style APIs that loads GGUF and safetensors weights.
92 engagement1 sourceredditFri, Sep 4, 2026, 09:15 AM
Releases·Tool Release
Reame: CPU-first LLM inference server built on llama.cpp released
Reame is a new LLM inference server designed to run efficiently on cheap CPU hardware, including shared vCPUs, free tiers, and 2-core ARM boxes. Built on llama.cpp, it features disk KV cache, self-regulating speculation, generation archive, and interleaved multi-user support. The project emphasizes treating CPU hardware as a first-class citizen rather than a fallback.
Reame is a new LLM inference server designed to run efficiently on cheap CPU hardware, including shared vCPUs, free tiers, and 2-core ARM boxes. Built on llama.cpp, it features disk KV cache, self-regulating speculation, generation archive, and interleaved multi-user support. The project emphasizes treating CPU hardware as a first-class citizen rather than a fallback.
60 engagement1 sourcegithubMon, Jul 6, 2026, 07:28 PM
Products·launchedfeatured
gemma4.c: 700-line C implementation of Gemma 4 E2B LLM
gemma4.c is a single-file C program that runs Google's Gemma 4 E2B language model on a CPU, implementing the tokenizer, transformer, KV cache, sampling, and CPU kernels from scratch. It allows developers to understand and run a modern LLM without external inference frameworks, making it educational and lightweight.
gemma4.c is a single-file C program that runs Google's Gemma 4 E2B language model on a CPU, implementing the tokenizer, transformer, KV cache, sampling, and CPU kernels from scratch. It allows developers to understand and run a modern LLM without external inference frameworks, making it educational and lightweight.
231 engagement2 sourcesredditThu, Aug 27, 2026, 11:53 PM
Products·launchedfeatured
Rust+CUDA LLM inference engine for RTX 5090, optimized with NVFP4, MoE, and speculative decoding
A from-scratch LLM inference engine written in Rust and CUDA, specifically tuned for the RTX 5090 Laptop GPU (Blackwell sm_120a). It supports NVFP4 quantization, mixture-of-experts (MoE), and multi-token prediction (MTP) speculative decoding, achieving up to 1.6x speedup over llama.cpp on target models. Designed for developers who need maximum inference performance on a single high-end laptop GPU.
A from-scratch LLM inference engine written in Rust and CUDA, specifically tuned for the RTX 5090 Laptop GPU (Blackwell sm_120a). It supports NVFP4 quantization, mixture-of-experts (MoE), and multi-token prediction (MTP) speculative decoding, achieving up to 1.6x speedup over llama.cpp on target models. Designed for developers who need maximum inference performance on a single high-end laptop GPU.
260 engagement1 sourcegithubSun, Jul 5, 2026, 08:45 PM