Skip to content

Built something? We create video reels & spotlights for GitHub projects.Promote your project →

Home / Topics / llm-serving

Topic Catalog

Top LLM-SERVING GitHub Repositories & Tools (2026)

Discover the most starred and trending open source tools tagged with #llm-serving.

SPONSORED BY

Kishan Kanani

Design & product building

Interviewly

Job search tools for developers

+ Advertise here

Your product in front of developers

+ Promote project

Get a video reel for your repo

5 matching projects · updated Sep 29, 2026 How we rank ↗

#1 TRENDING vllm-project/vllm Python 2023

A high-throughput and memory-efficient inference and serving engine for LLMs

◇ amd ◇ blackwell ◇ cuda ◇ deepseek

22.7K ↑ +14 today

#2 TRENDING ovg-project/kvcached Python NEW 2025

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

◇ elastic-kvcache ◇ gpu-mutiplexing ◇ gpu-sharing ◇ inference-engine

181 ↑ +1 today

#3 TRENDING pegainfer-project/pegainfer Rust NEW 2026

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2

◇ cuda ◇ cuda-kernels ◇ deepseek ◇ gpu

110 ↑ +1 today

#4 TRENDING ray-project/ray Python 2016

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

◇ data-science ◇ deep-learning ◇ deployment ◇ distributed

8.1K ↑ +1 today

#5 TRENDING alibaba/rtp-llm Python 2023

RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.

◇ gpt ◇ inference ◇ llama ◇ llm

278 ↑ +1 today

From our network

FOR MAINTAINERS

Built something? Put it in front of millions of developers.

We make a short reel about your project and post it across YouTube, Instagram, Threads, and X. Send a link, we do the rest.

Promote your project