a community oriented 1:1, vLLM-alike (Continuous batching, paged KV) engine in C++ with additional features (GGUF, RadixAttention, Cache-aware scheduling, ...)
↑ +4 today
Discover the best Continous Batching open source repositories on GitHub. Explore trending tools, live star counts, source code, and developer projects updated daily.