Synapse Open Synapse
AI

Using Q38FN with dflash2? Try lower draft model quants.

Reddit · r/LocalLLaMA · · 1 min read

Back with Qwen 3.8 27B i heard the advice that a Q4 draft model would produce roughly the same acceptance rate as a Q8 or higher. I have had my agent create a Q38FN Draft model in F16, Q8 and Q4, and the Q4 seemed to deliver the biggest speedup.

Latest from Reddit · r/LocalLLaMA