Training-free page-sparse decode in llama.cpp: Qwen3-30B-A3B is 1.72× faster at 128k on an RTX 5090, with a 1-point RULER loss
I've been working on a simple question: at long context, does decode really need to read the whole KV cache for every new token? I wanted something training-free that runs inside llama.cpp on normal GPUs, not a new model or a separate serving stack. So I built page-sparse decode into a fork.