Embeddings, a reranker and an LLM on one home GPU - what changed when I stopped running one container per model
My home RAG setup ran one container per model. There was an embedding server, a cross-encoder for reranking, an LLM server, and a small FastAPI service gluing them together, so five services in one compose file. Whenever two jobs overlapped, one of them got OOM-killed.