Synapse Open Synapse
AI

Olla v0.1.0 - Lightweight open-source LLM proxy for homelabs & on-prem inference (Ollama, vLLM, SGLang, llama.cpp, oMLX with failover, sticky sessions & load balancing)

Reddit · r/LocalLLaMA ·

Hey folks, About 18 months ago we started building Olla to solve some of the headaches we kept running into whilst managing multiple local LLM inference servers, both in our homelabs and at work. We've shared it here before when it was fairly early in development.

Latest from Reddit · r/LocalLLaMA