Distributed Inference GitHub Repositories (2026) · Open Source Tools

State of the art inference performance with modern accelerators on Kubernetes

FlashInfer: Kernel Library for LLM Serving

Peer-to-peer LLM inference in the browser: pool your devices to run big open models, for chat and coding agents.

High Performance Serving Runtime For Interactive Super Intelligence System

Best practices, reference architectures, and examples for distributed AI training and inference on AWS.

Reproducible Docker/vLLM deployment, tuning, and benchmarks for DeepSeek V4.1 Flash on a switchless three-node or four-node DGX Spark fabric.