Llamacpp GitHub Repositories (2026) · Open Source Tools

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.

Zero-overhead, terminal-native local-LLM orchestrator. Run any engine, any model via the same proxy.

Cross-platform on-device AI toolkit

User friendly GUI for configuring and launching llama.cpp

⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.

A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.

lcpp is a dart implementation of llama.cpp used by the mobile artificial intelligence distribution (maid)

Bonsai Demo

Instant, controllable, local pre-trained AI models in Rust

The easiest way to use Ollama in .NET

llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2

ML software (llama.cpp, ComfyUI, vLLM) builds for AMD gfx906 GPUs, e.g. Radeon VII / MI50 / MI60

Streamline your workflow with Lynkr, a CLI tool that acts as an HTTP proxy for efficient code interactions using Claude Code CLI.

High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.

llama.cpp/ik_llama.cpp launcher: loads big MoE models across mismatched multi-GPU rigs by exact VRAM math.

🚢 Yet another operator for running large language models on Kubernetes with ease. Powered by Ollama! 🐫

Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.

Synthetic Autonomic Mind - An AI assistant for everyone.

Running any GGUF SLMs/LLMs locally, on-device in Android