Releases·Tool Release
LLM API Speedtest: open-source Go CLI benchmarks LLM APIs and gateways on tokens/s, TTFB, latency, success rate
The open-source llm-api-speedtest project benchmarks LLM APIs, OpenAI-compatible gateways, and API proxies on token speed, TTFB, latency, and success rate. It produces reproducible JSON reports and supports scheduled runs via GitHub Actions. Practitioners can use it to compare providers on objective performance rather than marketing claims.
32 engagement1 sourceSun, Sep 6, 2026, 12:59 AM
AI Overview
- Go CLI with no third-party dependencies compares OpenAI-compatible APIs, gateways, and proxies on tokens/s, TTFB, total latency, and success rate
- Runs as a single binary or via scheduled GitHub Actions, outputting reproducible JSON reports; Chinese-language support included
Related
Social·Community
Fast at what? Measuring inference speed beyond the first token
A practitioner post argues that slow inference APIs can stem from three distinct bottlenecks — delayed first output, slow streaming, or long agent completion — each needing separate measurement. The post includes a disclosure that the author works with MangoBoost's agency and promotes Mango Inference as a serverless inference option.
A practitioner post argues that slow inference APIs can stem from three distinct bottlenecks — delayed first output, slow streaming, or long agent completion — each needing separate measurement. The post includes a disclosure that the author works with MangoBoost's agency and promotes Mango Inference as a serverless inference option.
5 engagement1 sourceredditThu, Oct 15, 2026, 01:58 PM
Releases·Tool Release
Developer releases smokebench, an open-source TUI for quick LLM benchmarking
A developer built smokebench, an open-source TUI for quickly benchmarking LLMs on local setups, addressing the gap between public leaderboards and real-world quantized or fine-tuned models. It supports any OpenAI or Anthropic compatible endpoint and tests across 8 categories including math, code, and creative writing.
A developer built smokebench, an open-source TUI for quickly benchmarking LLMs on local setups, addressing the gap between public leaderboards and real-world quantized or fine-tuned models. It supports any OpenAI or Anthropic compatible endpoint and tests across 8 categories including math, code, and creative writing.
2 engagement1 sourceredditFri, Aug 21, 2026, 09:55 AM
Products·launched
LLM provider benchmark comparison site comparing exact model IDs
ProviderBench is a free website that compares LLM providers on exact shared model IDs, showing response time, throughput, price, uptime, and routing tags. It solves the problem of provider leaderboards comparing different model catalogs, helping developers find the cheapest, fastest, and most reliable options for a given model.
ProviderBench is a free website that compares LLM providers on exact shared model IDs, showing response time, throughput, price, uptime, and routing tags. It solves the problem of provider leaderboards comparing different model catalogs, helping developers find the cheapest, fastest, and most reliable options for a given model.
6 engagement1 sourceredditThu, Jul 16, 2026, 01:45 PM
Products·launched
RouterDash: Free benchmark playground for OpenRouter LLMs
RouterDash is a free web tool that lets users compare and benchmark multiple LLMs side-by-side via OpenRouter. It solves the problem of testing model performance without writing custom scripts or switching between tabs, targeting developers and researchers who need quick performance comparisons.
RouterDash is a free web tool that lets users compare and benchmark multiple LLMs side-by-side via OpenRouter. It solves the problem of testing model performance without writing custom scripts or switching between tabs, targeting developers and researchers who need quick performance comparisons.
2 engagement1 sourceredditFri, Aug 7, 2026, 07:23 PM
Products·launched
OpenBenchmarks: open-source reproducible benchmarks for SaaS APIs
OpenBenchmarks provides open-source, reproducible benchmarks for SaaS APIs, starting with GTM APIs. It helps AI agents discover and evaluate the right APIs by providing standardized performance metrics, solving the problem of vendor selection in agentic workflows.
OpenBenchmarks provides open-source, reproducible benchmarks for SaaS APIs, starting with GTM APIs. It helps AI agents discover and evaluate the right APIs by providing standardized performance metrics, solving the problem of vendor selection in agentic workflows.
8 engagement1 sourcehackernewsSat, Jul 11, 2026, 08:50 PM