Synapse Open Synapse
AI

Qwen3.5-9B running on a PS4’s GPU, offline, using its own MTP head for speculative decoding

Reddit · r/LocalLLaMA ·

I made this, so a quick note: it’s my open-source project. Qwen3.5-9B runs on a stock PS4 at 31–39 tok/s. The MTP head that ships with Qwen3.5 made the biggest difference: it drafts up to two tokens and the full model checks them in one pass, which took it from 22 to 31–39 tok/s on this old GPU.

Latest from Reddit · r/LocalLLaMA