Synapse Open Synapse
AI

Betelgeuse running Qwen 3.8 Flash Next IQ4_XS at 24 tok/s on 2x8GB DDR4 (14 tok/s on a single 16GB stick) + RTX 3060 12GB — 0 dropped experts, 100% byte-exact

Reddit · r/LocalLLaMA · · 2 min read

In my last thread on Betelgeuse ( bel-cli / bel-server ), the main pushback in the comments was: "Okay cool, 22 tok/s on IQ2_XS , but 2-bit is too low—what happens on an actual 4-bit quant like IQ4_XS where the experts are almost 2x larger?"

Same story elsewhere

Latest from Reddit · r/LocalLLaMA