Synapse Open Synapse
AI

Squeezed a full RAG pipeline (BGE-M3 + reranker + TTS) into 4GB VRAM and 300MB RAM

Reddit · r/LocalLLaMA ·

been working on a local desktop reading app for the past few months and hit some insane hardware walls trying to get everything to run on my laptop. thought id share the architecture in case anyone else is trying to run heavy models on consumer hardware.

Latest from Reddit · r/LocalLLaMA