Open-source AI gateway for LLMs & AI agents, built in Rust. One OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock & more — routing, guardrails, caching, rate limits, observability.
Topic Catalog
Inference Gateway GitHub Repositories (2026) · Open Source Tools
Discover the best Inference Gateway open source repositories on GitHub. Explore trending tools, live star counts, source code, and developer projects updated daily.
SPONSORED BY
2 matching projects · updated Nov 21, 2026
How we rank ↗
↑ +19 today
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
↑ +1 today
From our network