Synapse Open Synapse
AI

llama.cpp: Add FP8 KV-cache

Reddit · r/LocalLLaMA · · 1 min read

Overview Add support for OCP-FP8-based KV-caches. Run-time, static per-tensor scales are supported, but optional. Additional information https://huggingface.co/nvidia/Mistral-Medium-3.5-128B-NVFP4 ...

Latest from Reddit · r/LocalLLaMA