Proposal: A community revolution for private quality benchmarks (and why endless chasing tok/s is rotting our brains
honestly we in r/LocalLLaMA need a revolution of people making their own private benchmarks(not tok/s) i mean benchmarking the quality(especially across quantisation). keep yours private and share the chart results to each other.