Consumer Gpu GitHub Repositories (2026) · Open Source Tools

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

llama.cpp fork for significantly improved performance on Ampere (especially RTX 3090 / 3090 Ti): example: 95+ tok/s over a 100K-token generation at temperature 1 for qwen3.8, supporting full 262K ctx. Supports low memory (12gb) configs for 200k+ ctx

An experimental local AI runtime. It relays one request through five small models with one job each, on a single 8 GB consumer GPU.