Synapse Open Synapse
AI

M4 Pro vs M5 Pro (24GB): Qwen 3.8 Flash Next IQ3 with MoEShift at 374 tok/s prefill, 14.7 tok/s decode, 8.5GB peak memory

Reddit · r/LocalLLaMA · · 1 min read

I benchmarked Qwen3.8-Flash-Next 125B ISTA-DASLab's GSQ-RCO IQ3_XXS quant (76GB on disk), on two Macs with 24GB unified memory using MoEShift backend: Metric M4 Pro (medians) M5 Pro (medians) Short prompt decode (72 input tokens) 9.84 tok/s 14.69 tok/s Long prompt prefill (6,060 input tokens)…

Latest from Reddit · r/LocalLLaMA