Synapse Open Synapse
Open source

Show HN: Warmctl – Fast cold starts for vLLM inference

Show HN · · 4 min read

GPU checkpoint/restore for vLLM: snapshot a warmed-up engine (compiled kernels, CUDA graphs, weights, 1-8 GPUs) into an OCI image and restore it in seconds instead of a multi-minute cold start. Sca...

Latest from Show HN