RustingStrata runs Flint 40B on a 12 GB card: 64 tok/s decode, 1.3x llama.cpp
I tested new trending Recursive Flint UD-Q5_K_XL (27.9 GiB) on an RTX 3060 12 GB with 30 GB RAM, comparing RustingStrata against llama.cpp using the same model, prompt, and generation settings.