Synapse Open Synapse
AI

Mica v0.5: The smartest 4B decision model is also the fastest one. 20 ms per answer, 25x faster than Jev

Reddit · r/LocalLLaMA ·

I'm sharing Mica v0.5, the follow-up to Mica v0.1. It's 4B and runs on one consumer GPU. On speed, it takes 19.8 ms per decision at 256 tokens and about 1.2 s at 32k on a single RTX 6000 Ada.

Latest from Reddit · r/LocalLLaMA