Kv Cache GitHub Repositories (2026) · Open Source Tools

LMCache: Supercharge Your LLM with the Fastest KV Cache Layer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

High Performance KV Cache for LLM Inference

Alibaba Cloud's high-performance KVCache system for LLM inference, with components for global cache management, inference simulation(HiSim), and more.

High-performance KV cache storage for LLM inference — GPU offloading, SSD caching, and cross-node sharing via RDMA. Works with vLLM and SGLang.