Synapse Open Synapse
AI

People who tried gpu-unlock on 3090s and such - does this make any difference on mixed-precision quants where you get some parts of the model quantized to <=Q8 but keep some tensors at BF16?

Reddit · r/LocalLLaMA · · 1 min read

Dunno, curious if there's any info out there.

Latest from Reddit · r/LocalLLaMA