Synapse Open Synapse
AI

Splitting up work to be split across mutiple slots on the same gpu for higher total tokens per second

Reddit · r/LocalLLaMA · · 1 min read

ok so i am trying to increase performance of llms i have running on my 7900 xtx via hipfire, i notice i get significantly higher total tokens per second when i run mutiple slots at once when running qwen 3.8 27b.

Latest from Reddit · r/LocalLLaMA