PromptDeck v1.1.0: open-source Tauri desktop app benchmarks local (Ollama, LM Studio) and cloud (OpenRouter, Groq, DeepSeek) LLMs side-by-side
The developer of PromptDeck released v1.1.0 of the open-source Tauri + React desktop app for prompting and benchmarking LLMs side-by-side, sharing it on r/LocalLLaMA. The new version adds cloud providers (OpenRouter, Groq, DeepSeek, Together, OpenAI, xAI and more) so local and cloud models can be compared in one run. It targets practitioners running local models via Ollama, LM Studio, and llama.cpp who want TPS/TTFT comparisons and scored test suites.
- Tauri + React app runs one prompt against up to 4 models simultaneously with live streaming, no telemetry or cloud calls unless a cloud key is added
- Compares TPS/TTFT with charts (marks estimated TPS when servers don't report usage) and runs 1–10 prompt Test Suites producing per-model win scoreboards
- v1.1 adds cloud providers: OpenRouter, Groq, DeepSeek, Together, OpenAI, xAI, Kimi, NVIDIA NIM, Ollama Cloud, Gemini, plus custom endpoints mixed with local models
- Includes winner votes, persistent leaderboard, diff view, SQLite history, PDF/Markdown/CSV export; binaries for Windows/macOS/Linux
- Caveats: cloud API keys stored in local storage (OS-keychain migration planned) and cloud-run cost tracking not yet implemented
Related
OpenBench: GUI for benchmarking Ollama models on local hardware
OpenBench is an open-source desktop tool that provides a graphical interface for benchmarking Ollama models on your own hardware. It tracks metrics like tokens/sec, TTFT, VRAM usage, GPU temperature, and context performance, and supports custom datasets and workloads. It also uses a larger local model as a judge to compare response quality, helping users choose the best model for their needs.
OpenBench is an open-source desktop tool that provides a graphical interface for benchmarking Ollama models on your own hardware. It tracks metrics like tokens/sec, TTFT, VRAM usage, GPU temperature, and context performance, and supports custom datasets and workloads. It also uses a larger local model as a judge to compare response quality, helping users choose the best model for their needs.
Developer releases smokebench, an open-source TUI for quick LLM benchmarking
A developer built smokebench, an open-source TUI for quickly benchmarking LLMs on local setups, addressing the gap between public leaderboards and real-world quantized or fine-tuned models. It supports any OpenAI or Anthropic compatible endpoint and tests across 8 categories including math, code, and creative writing.
A developer built smokebench, an open-source TUI for quickly benchmarking LLMs on local setups, addressing the gap between public leaderboards and real-world quantized or fine-tuned models. It supports any OpenAI or Anthropic compatible endpoint and tests across 8 categories including math, code, and creative writing.
Traliran AI Hub CLI: C11 TUI client for local and cloud LLMs
Traliran AI Hub CLI is a lightweight, zero-dependency terminal user interface (TUI) written in C11 with ncursesw and libcurl. It allows users to interact with local LLMs (Ollama, Llama.cpp) and cloud APIs (Groq, Gemini, DeepSeek, Claude) directly from the terminal, featuring parallel model benchmarking and comparison. It solves the problem of heavy, resource-intensive AI IDEs for terminal enthusiasts who prefer a fast, low-overhead environment.
Traliran AI Hub CLI is a lightweight, zero-dependency terminal user interface (TUI) written in C11 with ncursesw and libcurl. It allows users to interact with local LLMs (Ollama, Llama.cpp) and cloud APIs (Groq, Gemini, DeepSeek, Claude) directly from the terminal, featuring parallel model benchmarking and comparison. It solves the problem of heavy, resource-intensive AI IDEs for terminal enthusiasts who prefer a fast, low-overhead environment.
RouterDash — free prompt playground for comparing models across providers
RouterDash is a client-side prompt playground that lets users run the same prompt against models from multiple providers and compare responses. It helps developers test and evaluate model quality and cost, e.g. to find the cheapest high-quality option. All data stays in browser local storage, aimed at privacy-conscious users.
RouterDash is a client-side prompt playground that lets users run the same prompt against models from multiple providers and compare responses. It helps developers test and evaluate model quality and cost, e.g. to find the cheapest high-quality option. All data stays in browser local storage, aimed at privacy-conscious users.
I built LLM Speedtest — a free, open-source desktop app that benchmarks local LLMs with llama-bench-style test suites (Ollama, llama.cpp, vLLM, LM Studio…)
Developer releases LLM Speedtest, an open-source desktop app for benchmarking local LLM inference
Developer releases LLM Speedtest, an open-source desktop app for benchmarking local LLM inference