vllm-project

A high-throughput and memory-efficient inference and serving engine for LLMs

A framework for efficient model inference with omni-modality models

Cost-efficient and pluggable Infrastructure components for GenAI inference

Common recipes to run vLLM

vLLM plugin for attention-ffn disaggregation support