Unblocked describes routing LLM traffic across inference providers with TCP-style congestion control
Relay is a self-hosted gateway that routes LLM requests to different models based on evaluation results. It solves the problem of choosing the right model for each query by automatically falling back or switching models when quality thresholds are not met, useful for developers building production LLM applications.
Relay is a self-hosted gateway that routes LLM requests to different models based on evaluation results. It solves the problem of choosing the right model for each query by automatically falling back or switching models when quality thresholds are not met, useful for developers building production LLM applications.
Developer showcases Kairo, a fail-closed LLM inference router built from RTX 5090 measurements
Developer showcases Kairo, a fail-closed LLM inference router built from RTX 5090 measurements
Llmbridge is an LLM gateway/proxy written in C++ that routes requests to LLM providers with sub-millisecond added latency. It targets developers and teams serving LLM traffic who find existing gateways too slow or heavyweight. Posts show launch only; no features beyond latency claims are documented.
Llmbridge is an LLM gateway/proxy written in C++ that routes requests to LLM providers with sub-millisecond added latency. It targets developers and teams serving LLM traffic who find existing gateways too slow or heavyweight. Posts show launch only; no features beyond latency claims are documented.
A new open-source project called inference-cache provides a Kubernetes-native control plane for cache-aware LLM routing, aiming to reduce redundant computation of shared prefixes. It addresses a common inefficiency in serving repeated LLM prompts by leveraging cached prefixes.
A new open-source project called inference-cache provides a Kubernetes-native control plane for cache-aware LLM routing, aiming to reduce redundant computation of shared prefixes. It addresses a common inefficiency in serving repeated LLM prompts by leveraging cached prefixes.
A local control center that handles tasks deterministically first and falls back to LLM-based routing when needed. It targets developers who want reproducible automation without full dependence on AI. The LLM is used only for routing decisions the deterministic rules cannot resolve.
A local control center that handles tasks deterministically first and falls back to LLM-based routing when needed. It targets developers who want reproducible automation without full dependence on AI. The LLM is used only for routing decisions the deterministic rules cannot resolve.