Synapse Open Synapse
AI

MoE and expert offloading on llama, exllamav3, strata on 9070XT

Reddit · r/LocalLLaMA · · 2 min read

First off. I am by all means no expert, and excellent work has been done in the past few years that we all currently enjoy. Came over this post https://www.reddit.com/r/LocalLLM/comments/1x5mxcj/qwen3827b_on_a_single_rx_9070_xt_150_toks_on_code/ and wanted to try the engine on MoEs.

Latest from Reddit · r/LocalLLaMA