Synapse Open Synapse
AI

New sparse attention method for ultra-long context

Reddit · r/LocalLLaMA ·

I am on the team behind the attached open-source research on novel approaches to attention - specifically we show: • You can sparsify the attention of Qwen 3.8 27B without observed loss on accuracy • Extending context up to 10M tokens loses accuracy on some specific tasks - but most, like NIAH is…

Latest from Reddit · r/LocalLLaMA