Synapse Open Synapse
AI

llama.cpp: Possible to use Vulkan/iGPU and lazy-mode?

Reddit · r/LocalLLaMA · · 1 min read

With llama.cpp pure CPU mode and "--lazy-mode on" one can use larger than memory models, like Qwen3.8-Flash-Next and on-demand SSD streaming - slow but working. There is a nice iGPU with unified memory that can be used easily via Vulkan.

Latest from Reddit · r/LocalLLaMA