I built an engine that runs Qwen-Flash-Next at nearly 400t/s on cheap(ish) hardware. 3x V100 32GB in an old R740. Originally forked from strata but the engine guts all built for the old Volta generation.
CUDA engine that splits every layer of a production quality MoE LLM across three PCIe V100 cards delivering 382 tok/s on a used $1,800 box. - johnfink8/sfnx