llama.cpp: Add FP8 KV-cache
Overview Add support for OCP-FP8-based KV-caches. Run-time, static per-tensor scales are supported, but optional. Additional information https://huggingface.co/nvidia/Mistral-Medium-3.5-128B-NVFP4 ...
Overview Add support for OCP-FP8-based KV-caches. Run-time, static per-tensor scales are supported, but optional. Additional information https://huggingface.co/nvidia/Mistral-Medium-3.5-128B-NVFP4 ...