Top MULTIMODAL GitHub Repositories & Tools (2026)

Stop renting your intelligence. Own it with AnythingLLM. Everything you need for a powerful local-first agent experience

A native .NET LLM inference engine and agent runtime for GGUF models. TensorSharp provides a console application, a web-based chatbot interface, iPhone App, and Ollama/OpenAI-compatible HTTP APIs for programmatic access. It supports Windows/MacOS/iOS/Linux with full GPU capability

A framework for efficient model inference with omni-modality models

The Ruby-native AI framework. Chats, agents, tools, images, audio, and video through one consistent API, in plain Ruby or Rails.

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

Eyes for text-only DeepSeek Harness agents: built-in free vision chain (no key) + pixel-level vision tools (Q&A, grounding, crop, pixel diff, colors, OCR, SVG trace, cutout, screenshots). One-command install, no Python, image turns work like ordinary tool-calling turns.

An open-weight 11B model series for long-form and real-time video understanding

Curated systems, benchmarks, and papers etc. on memory for LLMs/MLLMs --- long-term context, retrieval, and reasoning.

An Asynchronous Reinforcement Learning Engine for Omni-Modal Post-Training at Scale

Knowledge-Aware machine LEarning (KALE): accessible machine learning from multiple sources for interdisciplinary research, part of the 🔥PyTorch ecosystem. ⭐ Star to support our work!

Open-source data management for multimodal AI. Query, trace, and govern with a lineage-native, format-agnostic lakehouse for agents and teams. Supports biological formats and registries - by the creators of Scanpy. 🍊YC S22

Real-time multimodal desktop agent evolving toward a persistent AI OS interface (0.15 α).

The backend agents build with - Multimodal database, orchestration, and serving in one file

Open-source adversarial testing engine, SDK, and CLI for AI agents. Runs locally or against the Humanbound Platform.

Streaming AI pipelines in one YAML file. Tokens, audio chunks, and video frames flow between isolated components. Compose 100+ components for models, agents, speech, vision, and live broadcast. Run local models, cloud APIs, or both. Inspired by docker-compose.

Auditable human-feedback annotation, review, provenance, and frozen training-data export.

Source-linked collection of published AI model benchmark scores.