SOTA model-agnostic quantization for GGUF, local inference and beyond. Quantize and run text, image, video and audio models with llama.cpp, vLLM, Diffusers and PyTorch.
Topic Catalog
Local Inference GitHub Repositories (2026) · Open Source Tools
Discover the best Local Inference open source repositories on GitHub. Explore trending tools, live star counts, source code, and developer projects updated daily.
SPONSORED BY
3 matching projects · updated Nov 21, 2026
How we rank ↗
↑ +147 today
Go With Your Own Intelligence! Use Go for hardware accelerated local inference with llama.cpp, whisper.cpp, and stable-diffusion.cpp directly integrated into your Go applications. Kronk provides a high-level API and production ready model server.
↑ +2 today
Run open decision models locally: pull and serve Laya, decider, NLI and GLiClass behind a TypeSafe-compatible API. Ollama for decision models.
↑ +1 today
From our network