110+ TPS through 100K tokens with 27b on a single 3090, with 400k ctx
Hello everyone! As always, thank you for sharing you previous results (and bugs) on the last version. I am happy to report that LlamAmpere v0.6 is now out.
Hello everyone! As always, thank you for sharing you previous results (and bugs) on the last version. I am happy to report that LlamAmpere v0.6 is now out.