SearchAI Inference Server v1.2.0 natively serves XHToken's Spark-X2.5 (4B/1.7B) — 1.7B decodes ~55 tok/s on a CPU

Products·launched

Pure WASM 4-bit LLM inference engine for browser, no GPU or server

A WebAssembly-based inference engine that runs a 350M-parameter LiquidAI LFM 2.5 model entirely in the browser with 4-bit quantization. It requires no GPU, no server, and no network at inference time, enabling private on-device AI chat for users without powerful hardware.

A WebAssembly-based inference engine that runs a 350M-parameter LiquidAI LFM 2.5 model entirely in the browser with 4-bit quantization. It requires no GPU, no server, and no network at inference time, enabling private on-device AI chat for users without powerful hardware.

Social·Community

Community discovers Spark-X2.5-4B and 1.7B small LLMs with native 1M context on Hugging Face

A Reddit user browsing Hugging Face found Spark-X2.5-4B and Spark-X2.5-1.7B from XHToken, small models with their own architecture rather than fine-tunes. The 4B reportedly benchmarks neck and neck with Qwen 3.5 9B, and both claim native 1M context. They do not yet run out of the box on llama.cpp.

A Reddit user browsing Hugging Face found Spark-X2.5-4B and Spark-X2.5-1.7B from XHToken, small models with their own architecture rather than fine-tunes. The 4B reportedly benchmarks neck and neck with Qwen 3.5 9B, and both claim native 1M context. They do not yet run out of the box on llama.cpp.

Products·launched

Reame: CPU-first inference server serving 7B MoE at 25 tok/s on free Oracle ARM

Reame is a llama.cpp-based CPU inference server that serves OLMoE (7B, 1B active MoE, Q4) at ~25 tok/s on a free Oracle ARM box. It demonstrates that MoE models can achieve high throughput on CPU, making large language model inference accessible at zero hosting cost.

Reame is a llama.cpp-based CPU inference server that serves OLMoE (7B, 1B active MoE, Q4) at ~25 tok/s on a free Oracle ARM box. It demonstrates that MoE models can achieve high throughput on CPU, making large language model inference accessible at zero hosting cost.

Developments·Benchmark

Qwen3.8-Flash-Next-oQ4e-mtp: 45 tok/s on M4 Max, 25 tok/s on M2 Ultra for local inference — llm-bench.io

llm-bench.io published local inference benchmarks for a quantized Qwen3.8-Flash-Next build on Apple Silicon, reporting 45 tok/s on an M4 Max and 25 tok/s on an M2 Ultra. The build is quantized to oQ4e with multi-token prediction and delivers speeds similar to Qwen3.8 27B on Apple Silicon.

llm-bench.io published local inference benchmarks for a quantized Qwen3.8-Flash-Next build on Apple Silicon, reporting 45 tok/s on an M4 Max and 25 tok/s on an M2 Ultra. The build is quantized to oQ4e with multi-token prediction and delivers speeds similar to Qwen3.8 27B on Apple Silicon.

A Dev Genius article reports that a single 4B multimodal model running on a free local CPU inference server—a 16-core, 32 GB RAM machine with no GPU—completed over 90% of enterprise AI tasks. The setup uses a one-line installation, suggesting practical viability for CPU-only deployments.