Releases·Tool Release
Orchestral 0.4 - a Rust agent runtime that reads model options from your provider
The maintainer of Orchestral, an open-source Rust agent runtime for local and remote models, released version 0.4.0 alongside Ferrum 0.11.0, a local LLM server. Orchestral 0.4 fixes stale model menus by refreshing model lists and reasoning capabilities from OpenAI-compatible providers, while Ferrum adds an int8 KV-cache dtype option. Practitioners running multi-provider or local setups get live model switching without waiting for agent releases.
8 engagement1 sourceFri, Sep 18, 2026, 12:44 AM
AI Overview
- Orchestral 0.4.0's /model menu refreshes provider model lists on OpenAI-compatible connections and preserves provider-defined effort names
- Ferrum 0.11.0 adds --kv-dtype int8, cutting reported KV allocation ~49% but slowing per-token time ~12% on an M1 Max with Qwen3.5-9B Q4_K_M
Entities
Related
Releases·Tool Release
Quartermaster: open source local AI platform, a llama-swap fork, auto-configures GGUF models behind one OpenAI-compatible API
A developer announced Quartermaster, an open source local AI platform that began as a fork of llama-swap and has since grown into its own project. Users point it at a models folder and it auto-configures runtime settings by reading GGUF headers and measuring free VRAM. It targets ease of use while keeping all settings editable per model.
A developer announced Quartermaster, an open source local AI platform that began as a fork of llama-swap and has since grown into its own project. Users point it at a models folder and it auto-configures runtime settings by reading GGUF headers and measuring free VRAM. It targets ease of use while keeping all settings editable per model.
44 engagement1 sourceredditThu, Sep 3, 2026, 07:36 PM
Releases·Tool Release
Row-Bot v4.8.0 adds provider-aware reasoning controls
Row-Bot v4.8.0 is released, introducing provider-aware reasoning controls that let each chat select a valid reasoning mode for the exact model in use. The update also improves context handling with safer custom endpoint assumptions and stronger rolling compaction safeguards.
Row-Bot v4.8.0 is released, introducing provider-aware reasoning controls that let each chat select a valid reasoning mode for the exact model in use. The update also improves context handling with safer custom endpoint assumptions and stronger rolling compaction safeguards.
3 engagement2 sourcesredditSat, Aug 22, 2026, 09:01 AM
Products·launched
agentc: sub-1 MB coding agent that drives Ollama via its OpenAI-compatible /v1 endpoint
agentc is a minimal, extensible coding agent that runs local models through Ollama's OpenAI-compatible /v1 endpoint. LLMs power its coding-agent loop, with Ollama as the default local backend. It solves the problem of a tiny, dependency-free local coding agent for developers running local models.
agentc is a minimal, extensible coding agent that runs local models through Ollama's OpenAI-compatible /v1 endpoint. LLMs power its coding-agent loop, with Ollama as the default local backend. It solves the problem of a tiny, dependency-free local coding agent for developers running local models.
5 engagement1 sourceredditThu, Oct 8, 2026, 03:28 AM
Releases·Tool Release
Millwright: open-source Rust-based LLM router launched
A developer released Millwright, a self-hosted LLM router written in Rust, offering a CLI interface for routing between OpenAI-compatible APIs, Anthropic, and Amazon Bedrock. It features policy-controlled model roles (cheap, mid, frontier) and cheapest healthy route selection, targeting cost savings and transparency for coding agents and GenAI workloads.
A developer released Millwright, a self-hosted LLM router written in Rust, offering a CLI interface for routing between OpenAI-compatible APIs, Anthropic, and Amazon Bedrock. It features policy-controlled model roles (cheap, mid, frontier) and cheapest healthy route selection, targeting cost savings and transparency for coding agents and GenAI workloads.
6 engagement1 sourcehackernewsWed, Jul 22, 2026, 07:03 PM
Releases·Tool Release
Developer launches Openheim, a Rust-based multi-provider LLM agent runtime
A developer posted Openheim on Hacker News, a runtime for running LLM agents across multiple providers, written in Rust. It matters to practitioners building agent applications who want a single runtime layer over different model vendors. Engagement was minimal, so community validation remains limited.
A developer posted Openheim on Hacker News, a runtime for running LLM agents across multiple providers, written in Rust. It matters to practitioners building agent applications who want a single runtime layer over different model vendors. Engagement was minimal, so community validation remains limited.
3 engagement1 sourcehackernewsTue, Sep 1, 2026, 01:54 PM