Synapse Open Synapse
AI

If you are using big model compared to ram try and disable mtp

Reddit · r/LocalLLaMA · · 1 min read

Little reminder, with all my testing I found out that mtp can decrease performance when using very big model, since many are starting to use strata or similar and I see many app activating it as default try and disable it, in my case, testing trough gyrostrata and strata with a 3060 and 32gb of ram both with gyro-m and swift 1.5 I got 20-30% improvement in tok/s just by disabling mtp.

Latest from Reddit · r/LocalLLaMA