Show HN: Sturnus, a latency-based LLM load balancer
I built sturnus because I wanted the latency-based load balancing of an AI gateway but without a big framework selling an enterprise plan. It’s a single static binary that does one thing, aiming to avoid the large surface area and security issues that have plagued some LLM gateways.