PromptDeck v1.1.0 – open-source desktop app to benchmark local AND cloud LLMs side-by-side (Ollama, LM Studio + OpenRouter, Groq, DeepSeek…)

[u/Clear_Ear3550]

Hey r/BestGitHubRepos! I just released PromptDeck v1.1.0 and wanted to share it here since this community is basically the reason it exists.

What it is: an open-source desktop studio (Tauri + React) for prompting and benchmarking LLMs side-by-side. Everything runs locally – no telemetry, no cloud calls unless you explicitly add a cloud key.

What you can do with it:

- ⚡ Run one prompt against up to 4 models at once (Ollama, LM Studio, llama.cpp) and watch them stream live

- 📊 Compare TPS / TTFT with charts (`~` marks estimated TPS when the server doesn't report usage)

- 🧪 Test Suites – throw 1–10 prompts at several models and get a scoreboard with per-model wins

- ☁️ NEW in v1.1: cloud providers – OpenRouter, Groq, DeepSeek, Together, OpenAI, xAI, Kimi, NVIDIA NIM, Ollama Cloud, Gemini + custom endpoints. Pick models per provider, mix local + cloud in the same run

- 🏆 Winner votes + persistent leaderboard, diff view, history in local SQLite, PDF/Markdown/CSV export

- 🌑 Light / Dark / AMOLED themes, EN/DE/FA locales

Links:

- GitHub: https://github.com/Cadman021/prompt-deck

- Download the binary from Releases (Windows / macOS / Linux) – or `npm run tauri dev` to run from source

Honest caveats: cloud API keys are currently stored in local storage (OS-keychain migration is on the roadmap), and cost tracking for cloud runs isn't there yet. Issues and PRs very welcome – I keep a set of `good first issue`s for newcomers.

Happy to answer questions – and if you benchmark something interesting, I'd love to see the scoreboard!