Show HN: A tool to benchmark LLM APIs (OpenAI, Claude, local/self-hosted)
Show HN (score: 5)Description
It runs a configurable number of test requests and reports two key metrics: • First-token latency (ms): How long it takes for the first token to appear • Output speed (tokens/sec): Overall output fluency
Demo: https://llmapitest.com/ Code: https://github.com/qjr87/llm-api-test
The goal is to provide a simple, visual, and reproducible way to evaluate performance across different LLM providers, including the growing number of third-party “proxy” or “cheap LLM API” services.
It supports: • OpenAI-compatible APIs (official + proxies) • Claude (via Anthropic) • Local endpoints (custom/self-hosted)
You can also self-host it with docker-compose. Config is clean, adding a new provider only requires a simple plugin-style addition.
Would love feedback, PRs, or even test reports from APIs you’re using. Especially interested in how some lesser-known services compare.
More from Show
Show HN: Pi-hosts – Give the Pi coding agent access to your servers
Show HN: Pi-hosts – Give the Pi coding agent access to your servers I built that initially for an AI chat bot that allows teams to perform DevOps tasks straight out of Slack/Teams (with proper permission control, obviously).<p>Useful to let developers perform mundane tasks, or help coordinate incident response.<p>I ended up using it myself on my own machine to manage Hetzner and AWS boxes. I thought that may be useful to others.<p>The default policy should be good enough, but if you don't like living dangerously, you can set it to `paranoid`.<p>Install: `pi install npm:pi-hosts`<p>GitHub: <a href="https://github.com/hunvreus/pi-hosts" rel="nofollow">https://github.com/hunvreus/pi-hosts</a>
Show HN: 49Agents – 2D Canvas IDE for Orchestrating Agents, Repos, Issues
Show HN: 49Agents – 2D Canvas IDE for Orchestrating Agents, Repos, Issues Beads tables (Steve Yegge's) for issue tracking. Can view git trees, terminals, issue tables, notes, and files all on one screen. Can connect multiple machines via private network (like tailscale)
Show HN: ClusterdOS – Kubernetes without the platform team
Show HN: ClusterdOS – Kubernetes without the platform team
No other tools from this source yet.