Synapse Open Synapse
AI

RustingStrata runs Flint 40B on a 12 GB card: 64 tok/s decode, 1.3x llama.cpp

Reddit · r/LocalLLaMA · · 1 min read

I tested new trending Recursive Flint UD-Q5_K_XL (27.9 GiB) on an RTX 3060 12 GB with 30 GB RAM, comparing RustingStrata against llama.cpp using the same model, prompt, and generation settings.

Latest from Reddit · r/LocalLLaMA