Synapse Open Synapse
AI

vLLM now has PLE disk offload support on nightly

Reddit · r/LocalLLaMA · · 3 min read

Overview Adds --engram-config '{"disk_offload": true}' for Qwen4Exp. The PLE n-gram table stays in the checkpoint files, and each step reads only its rows on the host. On DGX Spar...

Latest from Reddit · r/LocalLLaMA