Synapse Open Synapse
AI

[Paper] FlashMask-3: Efficient and Expressive Mask-Aware Distributed Attention

Reddit · r/LocalLLaMA · · 1 min read

Long-context training of large language and multimodal models couples structured attention masks (document, prefix-LM, share-question, sliding-window, block-sparse) with multi-node context parallelism, making the mask a dominant bottleneck.

Latest from Reddit · r/LocalLLaMA