Synapse Open Synapse
AI

Llama.cpp setup Qwen-flash-next Vulkan Q4 with Expert cache.

Reddit · r/LocalLLaMA · · 1 min read

I just setup my server: 64GB DDR4 2133 4CH (Broadwell Xeon) 2 x RX 6800 16GB 1 x Rx 580 2048sp (16GB) I am using the Q4 of Qwen Flash Next, with the prompt: sudo ./llama-server -m ./models/Qwnflash180b-Q4.gguf --host 0.0.0.0 --port 8081 -ngl 99 -fa on --moe-cache-mib 16384 --log-verbosity 4…

Latest from Reddit · r/LocalLLaMA