Just update of ik_llama.cpp fork and seek of feedback
Continuation of https://www.reddit.com/r/LocalLLaMA/comments/1wynomx/k_llamacpp_moe_optimizations_expert_residency/ This time I finally get speeds compared to Strata for ik_llama.cpp on 6gb vram rtx 2060 ( generates about 45–48 t/s on short prompts and about 37 t/s at 28K context.