Interpretability GitHub Repositories (2026) · Open Source Tools

Steering-first neuron explorer for Qwen3.5-4B: push every MLP neuron, keep every rollout, browse the results

open source interpretability platform 🧠

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

Fit interpretable models. Explain blackbox machine learning.

Laplace's Scissors: protected readout rotations for open LLMs. Diagnose weak output directions, repair answers, preserve successful generations, and analyze scaling deviations with Rust and Triton on CUDA/HIP.

A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.