Releases·Tool Release
Show HN: Compute:Arena – Community submitted local AI benchmarks
The Compute:Arena team open-sourced their internal testing harness, letting anyone benchmark open source models on their own hardware and submit results to a public leaderboard. The site is live with a few hundred submissions already, covering combinations of open models, quantizations, and chips from AMD, NVIDIA, Apple Silicon, Intel, and Qualcomm. It gives practitioners a practical way to check real-world model-on-hardware performance.
7 engagement1 sourceThu, Sep 17, 2026, 06:54 AM
AI Overview
- Compute:Arena open-sourced its internal testing harness so anyone can benchmark open source models across AMD, NVIDIA, Apple Silicon, Intel, and Qualcomm hardware.
- The public leaderboard is live with a few hundred community submissions already.
- Submitters can file hardware-specific benchmark results for hundreds of open source models and quantizations.
Related
Releases·Tool Release
Distributed Local Agents Benchmark
A developer launched airbench.ai, a website where anyone can run and share benchmark tests comparing agent harnesses against local models across different hardware. The goal is a fully distributed agent benchmark, since exhaustively testing every harness-model-hardware combination exceeds any single person's capacity. It includes a leaderboard the creator hopes others will populate with their own tests.
A developer launched airbench.ai, a website where anyone can run and share benchmark tests comparing agent harnesses against local models across different hardware. The goal is a fully distributed agent benchmark, since exhaustively testing every harness-model-hardware combination exceeds any single person's capacity. It includes a leaderboard the creator hopes others will populate with their own tests.
1 engagement1 sourceredditTue, Sep 29, 2026, 11:48 AM
Releases·Tool Release
Developer builds Sakura, a reproducible benchmark for local coding models on Ollama
A developer created Sakura, a benchmark that runs 27 coding tasks against any Ollama model to measure accuracy, latency, and throughput, replacing subjective 'vibes' with reproducible numbers. It runs in a sandboxed Docker container and auto-detects hardware, making it usable on laptops with limited VRAM.
A developer created Sakura, a benchmark that runs 27 coding tasks against any Ollama model to measure accuracy, latency, and throughput, replacing subjective 'vibes' with reproducible numbers. It runs in a sandboxed Docker container and auto-detects hardware, making it usable on laptops with limited VRAM.
1 engagement1 sourceredditWed, Aug 26, 2026, 05:10 PM
Social·Community
Harness x model combination: more data! (airbench.ai crowdsources best configs)
A Reddit user launched airbench.ai, a site where people test and share their best local LLM harness-model-hardware combinations. Crowdsourced data already surfaced a winning config: a stranger beat the creator's 5090 baseline using a 3090. The site added a contributor section and reveals which harness performs best per model.
A Reddit user launched airbench.ai, a site where people test and share their best local LLM harness-model-hardware combinations. Crowdsourced data already surfaced a winning config: a stranger beat the creator's 5090 baseline using a 3090. The site added a contributor section and reveals which harness performs best per model.
1 engagement1 sourceredditThu, Oct 8, 2026, 11:52 AM
Releases·Tool Release
Developer releases smokebench, an open-source TUI for quick LLM benchmarking
A developer built smokebench, an open-source TUI for quickly benchmarking LLMs on local setups, addressing the gap between public leaderboards and real-world quantized or fine-tuned models. It supports any OpenAI or Anthropic compatible endpoint and tests across 8 categories including math, code, and creative writing.
A developer built smokebench, an open-source TUI for quickly benchmarking LLMs on local setups, addressing the gap between public leaderboards and real-world quantized or fine-tuned models. It supports any OpenAI or Anthropic compatible endpoint and tests across 8 categories including math, code, and creative writing.
2 engagement1 sourceredditFri, Aug 21, 2026, 09:55 AM
Products·launched
Benchmark local LLMs against your device specs
A tool that benchmarks local LLMs using custom task sets via Ollama and similar providers, evaluating on metrics with deterministic criteria or LLM judges. It queries HuggingFace to compare trending models with your device's specs, helping users identify which models are worth testing on their hardware.
A tool that benchmarks local LLMs using custom task sets via Ollama and similar providers, evaluating on metrics with deterministic criteria or LLM judges. It queries HuggingFace to compare trending models with your device's specs, helping users identify which models are worth testing on their hardware.
1 engagement1 sourcehackernewsFri, Aug 7, 2026, 09:57 AM