Vlm GitHub Repositories (2026) · Open Source Tools

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

🚀🚀 A multimodal System One decision model that gives calibrated answers to typed questions about screens, photos, video and text in one forward pass.

SGLang is a high-performance serving framework for large language models and multimodal models.

An open-weight 11B model series for long-form and real-time video understanding

A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

TokenSpeed is a speed-of-light LLM inference engine.

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

NVIDIA AI Blueprint for video search and summarization (VSS) is a GPU-accelerated reference architecture for building video analytics agents with real-time verified alerts, visual Q&A, and automated reporting. The VSS Blueprint uses vision language models (VLMs) such as NVIDIA Cosmos, LLMs such as NVIDIA Nemotron, RAG, and NVIDIA NIMs.

Swift SDK for running chat, vision and speech models on iPhone and Mac with Apple's Core AI. Model download and caching, FoundationModels integration, and runnable examples with documented OS, SDK and model requirements.

Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code

llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2

The Agentic Operating System for Robots

Efficient LLM inference on Slurm clusters.

Node Creative Coding / 3D / Image Processing tool inspired by Quartz Composer