Synapse Open Synapse
AI

For long context does high quant matter?

Reddit · r/LocalLLaMA · · 1 min read

Question: if I'm regularly at higher context - say 300-400k - do higher quant versions of the same model perform better? I'm thinking something like Qwen Flash Next q6 and q8. I recently swapped from q6 to q8 and performed a/b testing.

Latest from Reddit · r/LocalLLaMA