Developer Pavel launched Relay, a self-hostable AI gateway that classifies incoming LLM requests and routes them across configured providers while pacing against rate limits. It targets teams juggling multiple providers' free tiers and shared agent capacity. A community version is available now alongside a hosted tier.
Relay is a self-hosted gateway that routes LLM requests to different models based on evaluation results. It solves the problem of choosing the right model for each query by automatically falling back or switching models when quality thresholds are not met, useful for developers building production LLM applications.
Relay is a self-hosted gateway that routes LLM requests to different models based on evaluation results. It solves the problem of choosing the right model for each query by automatically falling back or switching models when quality thresholds are not met, useful for developers building production LLM applications.
A developer on Reddit is looking for an LLM router that optimizes for throughput by managing provider rate limits (TPM/RPM) rather than model quality. They use both OpenAI and AWS Bedrock and want a token-meter-aware system to configure multiple equivalent endpoints for the same task.
A developer on Reddit is looking for an LLM router that optimizes for throughput by managing provider rate limits (TPM/RPM) rather than model quality. They use both OpenAI and AWS Bedrock and want a token-meter-aware system to configure multiple equivalent endpoints for the same task.
Phlox-GW is an open-source LLM gateway that provides routing, rate limiting, and observability for multiple LLM providers without the high cost of enterprise solutions. It solves the problem of managing and securing access to LLMs for developers and small teams who need a lightweight, self-hosted alternative.
Phlox-GW is an open-source LLM gateway that provides routing, rate limiting, and observability for multiple LLM providers without the high cost of enterprise solutions. It solves the problem of managing and securing access to LLMs for developers and small teams who need a lightweight, self-hosted alternative.
A developer released Millwright, a self-hosted LLM router written in Rust, offering a CLI interface for routing between OpenAI-compatible APIs, Anthropic, and Amazon Bedrock. It features policy-controlled model roles (cheap, mid, frontier) and cheapest healthy route selection, targeting cost savings and transparency for coding agents and GenAI workloads.
A developer released Millwright, a self-hosted LLM router written in Rust, offering a CLI interface for routing between OpenAI-compatible APIs, Anthropic, and Amazon Bedrock. It features policy-controlled model roles (cheap, mid, frontier) and cheapest healthy route selection, targeting cost savings and transparency for coding agents and GenAI workloads.
Foreman is a self-hosted LLM gateway that routes requests between coding agents and multiple providers, keeping keys and traffic within the user's network. It enables cost tracking, model switching without code changes, and policy-based routing to cheaper models.
Foreman is a self-hosted LLM gateway that routes requests between coding agents and multiple providers, keeping keys and traffic within the user's network. It enables cost tracking, model switching without code changes, and policy-based routing to cheaper models.