Synapse Open Synapse
AI

How much code do AI models and harnesses add? I made an open-source benchmark to measure code complexity.

Reddit · r/LocalLLaMA ·

Existing benchmarks measure the solve rate and token output/cost, but don't punish excess code or overengineering, which increases technical debt in the long term. Parsimony counts syntax units (statements, calls, names) in the patches other benchmarks publish.

Latest from Reddit · r/LocalLLaMA