Synapse Open Synapse
AI

Training-free page-sparse decode in llama.cpp: Qwen3-30B-A3B is 1.72× faster at 128k on an RTX 5090, with a 1-point RULER loss

Reddit · r/LocalLLaMA · · 3 min read

I've been working on a simple question: at long context, does decode really need to read the whole KV cache for every new token? I wanted something training-free that runs inside llama.cpp on normal GPUs, not a new model or a separate serving stack. So I built page-sparse decode into a fork.

Latest from Reddit · r/LocalLLaMA