Developer showcases Kairo, a fail-closed LLM inference router built from RTX 5090 measurements
A developer released open-source AMDGCN kernels aimed at optimizing LLM inference. The kernels target AMD hardware to improve performance.
A developer released open-source AMDGCN kernels aimed at optimizing LLM inference. The kernels target AMD hardware to improve performance.
A developer published a time-to-first-token (TTFT) benchmark comparing LLM Gateway and OpenRouter using Claude Haiku 4.5 over 150 runs. The results provide practitioners with concrete latency data for choosing between these two LLM API gateways.
A developer published a time-to-first-token (TTFT) benchmark comparing LLM Gateway and OpenRouter using Claude Haiku 4.5 over 150 runs. The results provide practitioners with concrete latency data for choosing between these two LLM API gateways.
Relay is a self-hosted gateway that routes LLM requests to different models based on evaluation results. It solves the problem of choosing the right model for each query by automatically falling back or switching models when quality thresholds are not met, useful for developers building production LLM applications.
Relay is a self-hosted gateway that routes LLM requests to different models based on evaluation results. It solves the problem of choosing the right model for each query by automatically falling back or switching models when quality thresholds are not met, useful for developers building production LLM applications.
A GitHub repository providing minimal, reproducible post-training experiments (SFT, DPO, GRPO) for LLMs that run on a single 8GB GPU. It aims to help researchers and hobbyists explore post-training techniques without expensive hardware.
A GitHub repository providing minimal, reproducible post-training experiments (SFT, DPO, GRPO) for LLMs that run on a single 8GB GPU. It aims to help researchers and hobbyists explore post-training techniques without expensive hardware.
A new open-source project called inference-cache provides a Kubernetes-native control plane for cache-aware LLM routing, aiming to reduce redundant computation of shared prefixes. It addresses a common inefficiency in serving repeated LLM prompts by leveraging cached prefixes.
A new open-source project called inference-cache provides a Kubernetes-native control plane for cache-aware LLM routing, aiming to reduce redundant computation of shared prefixes. It addresses a common inefficiency in serving repeated LLM prompts by leveraging cached prefixes.