Synapse Open Synapse
AI

MoE Offloading update Exllamav3 on 9070XT

Reddit · r/LocalLLaMA · · 1 min read

This is a follow up on https://www.reddit.com/r/LocalLLaMA/comments/1xun15v/moe_and_expert_offloading_on_llama_exllamav3/ While we wait for Qwen 4, here is Qwen 3.6 35B A3B at Q4 on 9070XT 16GB with offloaded experts on DDR4 3200Mhz ram.

Latest from Reddit · r/LocalLLaMA