Blackwell GitHub Repositories (2026) · Open Source Tools

A high-throughput and memory-efficient inference and serving engine for LLMs

OpenLake is a high performance storage engine for efficient LLM inference and GPU Training

SGLang is a high-performance serving framework for large language models and multimodal models.

TokenSpeed is a speed-of-light LLM inference engine.

Strata fork: Qwen3.8-Flash-Next 125B MoE in NVFP4 on one RTX 20-50 card (12 GB+) and 64 GB of RAM or more. Setup installs our GPTQ quants (huihui abliterated, OrcaRouter) from Hugging Face. CPU experts on AVX-512 or AVX2 (+AVX-VNNI), all experts in RAM from 92 GB, a low-RAM mode below, vision, W4A8 prompts. Windows / CUDA 13.

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

Run large AI chat models privately on your own NVIDIA RTX 50 or RTX PRO graphics card. On one RTX 5090, Qwen3.8-27B writes about 70 words per second, 1.4x faster than vLLM. Even a model almost four times larger than the card's memory runs. Works with apps made for ChatGPT or Claude. Starts with one docker command.