Top VLM GitHub Repositories & Tools (2026)

SGLang is a high-performance serving framework for large language models and multimodal models.

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support

TokenSpeed is a speed-of-light LLM inference engine.

An open-weight 11B model series for long-form and real-time video understanding

Node Creative Coding / 3D / Image Processing tool inspired by Quartz Composer

Swift SDK for running chat, vision and speech models on iPhone and Mac with Apple's Core AI. Model download and caching, FoundationModels integration, and runnable examples with documented OS, SDK and model requirements.