Synapse Open Synapse
AI

Rented some server time and wanted to share some learnings from benching using RAM heavier servers with some decent GPU: AMD EPYC and ROME motherboards really *ARE* worth it: Qwen 3.8 Flash Next Unsloth Q8 at 94 tok/s c=1, 456 tok/s c=8 .

Reddit · r/LocalLLaMA · · 1 min read

Raw data in the comments, if you go for that sort of thing. The TL;DR This build generates a nearly linear performance per stream up to at least 8 streams using a context window of 124k. I didn't test beyond that. Abstract Until today, I considered Strata a toy. I don't anymore.

Latest from Reddit · r/LocalLLaMA