Synapse Open Synapse
AI

Rust inference engine for Qwen models on a 12 GB GPU, and it runs Qwen3.8-Flash-Next faster than the original Strata

Reddit · r/LocalLLaMA · · 3 min read

I've been working on this for a while: RustingStrata. It's an inference engine for Qwen GGUF models, written in pure Rust, no ggml. The CUDA kernels get compiled at startup with NVRTC, and the same kernels also run on Vulkan.

Latest from Reddit · r/LocalLLaMA