Synapse Open Synapse
AI

Two days tuning a single Tesla P40: 35B MoE, full 262k context, ~60 t/s short and ~30 t/s at 100k deep. No sub-Q4 quants.

Reddit · r/LocalLLaMA · · 2 min read

i Got a bit obsessed with my P40 this week trying to max speed on Ornith 1.5 35B-A3B (qwen3.5 MoE arch) without going below Q4 weights or q8 KV.

Latest from Reddit · r/LocalLLaMA