Synapse Open Synapse
Open source

Two GPUs, multiple models, which ones should get a dedicated card?

Reddit · r/selfhosted · · 1 min read

We’re a team of 3 sharing one box with two GPUs, like 24GB card and a 16GB card and we need to run several models: a couple of 7B ones plus a few smaller low-traffic ones. I’m considering to try GPUStack for unified deployment and GPU allocation, and real question is how to split the two cards.

Latest from Reddit · r/selfhosted