Releases·Tool Release
[Discussion] PULSAR-ASM: 5.2KB pure x86-64 assembly inference engine for Gemma-2B (FP16, ~4.6 tok/s on CPU)
A developer shared PULSAR-ASM, an open-source inference engine for Gemma-2B written entirely in flat x86-64 assembly (FASM) with no C/C++ runtime or PyTorch. The 5.2 KB binary runs FP16 decoding at ~4.6 tokens/s on an older quad-core i5 CPU. A follow-up post used a residual-stream probe to show the model answering a factual question correctly through Layer 14, then outputting a wrong answer from Layer 15.
38 engagement2 sourcesSun, Oct 4, 2026, 03:48 AM
Developments
Follow-up post traces a Layer 15 hallucination: residual-stream truth probe flips sign before the engine outputs an incorrect '15%' VAT answer.
Tue, Oct 6, 2026, 03:00 AM
AI Overview
- Developer tomtsai28's open-source PULSAR-ASM totals 5.2KB flat machine code (3.7KB engine + 1.5KB AVX2/F16C GEMM) with zero C/C++ runtime or PyTorch.
- Pure AVX2+F16C with custom 4-thread SMP GEMM sustains ~18.5 GB/s memory bandwidth on DDR4-2400, decoding 4.5-4.7 tok/s FP16 on a quad-core i5.
- Author positions it as a first-principles reference for micro-LLMs on constrained MCUs/DSPs, not a llama.cpp competitor; Python harness only uses ctypes for VirtualAlloc and OS threads.
Entities
Related
Releases·Tool Release
Project Zero: from-scratch C99 BitNet inference engine beats bitnet.cpp 1.8× on Xeon, seeks testers
A developer released Project Zero, a pure C99 CPU-only inference engine for BitNet b1.58-2B-4T and Qwen Bonsai-27B, achieving 36 tok/s on a Xeon — 1.8× faster than bitnet.cpp. The single-binary, zero-dependency tool is open-source and seeking community benchmarks.
A developer released Project Zero, a pure C99 CPU-only inference engine for BitNet b1.58-2B-4T and Qwen Bonsai-27B, achieving 36 tok/s on a Xeon — 1.8× faster than bitnet.cpp. The single-binary, zero-dependency tool is open-source and seeking community benchmarks.
69 engagement1 sourceredditWed, Jul 22, 2026, 07:02 PM
Products·building
SHADOW-50M: a 44M-parameter LLM trained from scratch, 19.8 MB, ~1,900 tok/s on CPU
A small language model trained from scratch on 45B tokens, quantized to ternary weights so the full model is 19.8 MB and runs on a laptop CPU at ~1,900 tokens/s. It targets people running LLMs locally on constrained hardware, improving on the author's earlier SHADOW-250M in speed and size.
A small language model trained from scratch on 45B tokens, quantized to ternary weights so the full model is 19.8 MB and runs on a laptop CPU at ~1,900 tokens/s. It targets people running LLMs locally on constrained hardware, improving on the author's earlier SHADOW-250M in speed and size.
234 engagement1 sourceredditTue, Sep 15, 2026, 12:59 PM
Releases·Tool Release
From-scratch nucleo engine beats llama.cpp on ARM for PrismML's ternary Bonsai-8B
A developer built nucleo, a from-scratch inference engine, to run PrismML's ternary Bonsai-8B on Oracle's free 2-core ARM instance, outperforming the official llama.cpp fork by 14% in decode and 55% in prefill. The engine is integrated into Reame, a server for minimal hardware, with a live demo.
A developer built nucleo, a from-scratch inference engine, to run PrismML's ternary Bonsai-8B on Oracle's free 2-core ARM instance, outperforming the official llama.cpp fork by 14% in decode and 55% in prefill. The engine is integrated into Reame, a server for minimal hardware, with a live demo.
12 engagement1 sourceredditFri, Aug 21, 2026, 07:24 PM
Social·Community
I designed a CPU-native LLM architecture that hits 100+ tok/s on a 10B parameter model (the quality is the problem)
Reddit user WildPino posted SiliconLLM, a CPU-native LLM architecture designed around memory constraints. A 10B-parameter model runs at 113-130 tok/s on a Ryzen 5 3600X with no GPU, but the author acknowledges output quality remains poor. They are attempting to adapt Qwen2.5-Coder into the SSM/ternary/sparse format while training a small LM to validate quality at scale.
Reddit user WildPino posted SiliconLLM, a CPU-native LLM architecture designed around memory constraints. A 10B-parameter model runs at 113-130 tok/s on a Ryzen 5 3600X with no GPU, but the author acknowledges output quality remains poor. They are attempting to adapt Qwen2.5-Coder into the SSM/ternary/sparse format while training a small LM to validate quality at scale.
8 engagement1 sourceredditTue, Sep 15, 2026, 03:43 PM
Products·building
CPU-only LLM inference engine for mobile devices
A lightweight (450kb) inference engine built from scratch that runs LLMs like LFM2.5-2.6B on a OnePlus 13 at 17 tok/s using only CPU. It supports multiple model architectures and includes a device probe suite for performance monitoring via ADB.
A lightweight (450kb) inference engine built from scratch that runs LLMs like LFM2.5-2.6B on a OnePlus 13 at 17 tok/s using only CPU. It supports multiple model architectures and includes a device probe suite for performance monitoring via ADB.
235 engagement1 sourceredditWed, Aug 5, 2026, 02:20 PM