Two GPUs, multiple models, which ones should get a dedicated card?
We’re a team of 3 sharing one box with two GPUs, like 24GB card and a 16GB card and we need to run several models: a couple of 7B ones plus a few smaller low-traffic ones. I’m considering to try GPUStack for unified deployment and GPU allocation, and real question is how to split the two cards.