Releases·Tool Release
Redis creator launches ds4, a tool for running LLMs locally
From the creator of Redis: ds4 runs LLMs locally
402 engagement1 sourceFri, Oct 2, 2026, 06:01 PM
AI Overview
- ds4, built by the creator of Redis, lets users run LLMs locally; details at dwarfstar.sh
- The launch drew 44 points on Hacker News on 2026-10-02
Related
Releases·Tool Release
DeepSeek V4.1F Q4 on M3 Ultra with native DSpark MTP (40tps / 800tps)
A developer forked antirez/ds4 to optimize DeepSeek V4.1 Flash Q4 inference on Apple's M3 Ultra, roughly doubling decode speed and adding the first DSpark multi-token-prediction implementation on Metal. The fork preserves byte-identical output under greedy decode and is published with verification manifests on GitHub.
A developer forked antirez/ds4 to optimize DeepSeek V4.1 Flash Q4 inference on Apple's M3 Ultra, roughly doubling decode speed and adding the first DSpark multi-token-prediction implementation on Metal. The fork preserves byte-identical output under greedy decode and is published with verification manifests on GitHub.
58 engagement1 sourceredditTue, Sep 15, 2026, 11:51 AM
Products·launched
SSD-streamed MoE inference for Qwen3.6-35B-A3B on 16GB M1 Pro
A modified runtime (ds4) that runs the Qwen3.6-35B-A3B mixture-of-experts model on a 16 GB M1 Pro by streaming expert weights from SSD. It keeps a bounded cache of routed experts in unified memory and loads misses from disk, enabling inference on hardware that cannot hold the full model. Targets developers who want to run large MoE models locally on memory-constrained Apple Silicon machines.
A modified runtime (ds4) that runs the Qwen3.6-35B-A3B mixture-of-experts model on a 16 GB M1 Pro by streaming expert weights from SSD. It keeps a bounded cache of routed experts in unified memory and loads misses from disk, enabling inference on hardware that cannot hold the full model. Targets developers who want to run large MoE models locally on memory-constrained Apple Silicon machines.
2 engagement2 sourceshackernews, redditFri, Jul 17, 2026, 11:20 PM
Social·Community
Users report KV cache checkpoint misses with multiple tool calls
A user reports that when a model makes many tool calls in one turn, KV cache checkpoints created around those calls become invalid and are discarded, forcing full re-processing. This behavior is observed across llama.cpp, ds4, and various frontends like Open WebUI, OpenClaw, and Codex. The issue appears to be triggered by multiple tool calls, while single or double calls do not cause it.
A user reports that when a model makes many tool calls in one turn, KV cache checkpoints created around those calls become invalid and are discarded, forcing full re-processing. This behavior is observed across llama.cpp, ds4, and various frontends like Open WebUI, OpenClaw, and Codex. The issue appears to be triggered by multiple tool calls, while single or double calls do not cause it.
9 engagement1 sourceredditSun, Aug 2, 2026, 11:47 PM
Releases·Tool Release
Show HN: mlxsh A lightweight CLI to serve multiple local LLMs on Apple
A developer released mlxsh, a lightweight CLI that serves multiple local LLMs on Apple hardware. It leverages the MLX framework to run models efficiently on Apple Silicon.
A developer released mlxsh, a lightweight CLI that serves multiple local LLMs on Apple hardware. It leverages the MLX framework to run models efficiently on Apple Silicon.
1 engagement1 sourcehackernewsMon, Aug 3, 2026, 12:03 AM
Releases·Tool Release
Kern Sandbox runs LLM-generated code locally with no cloud or daemon
A developer launched Kern Sandbox on Hacker News, a tool that runs LLM-generated code locally without cloud services or a background daemon. It targets practitioners executing AI-written code in an isolated environment.
A developer launched Kern Sandbox on Hacker News, a tool that runs LLM-generated code locally without cloud services or a background daemon. It targets practitioners executing AI-written code in an isolated environment.
4 engagement1 sourcehackernewsMon, Sep 28, 2026, 01:01 PM