Arrow GitHub Repositories (2026) · Open Source Tools

An open-source inference engine for AI functions in SQL

Extremely fast Query Engine for DataFrames, written in Rust

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

Apache DataFusion SQL Query Engine

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics

Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.

Apache DataFusion Ballista Distributed Query Engine

Library for bringing distributed capabilities to Apache DataFusion

Ergonomic bindings to duckdb for Rust

Official Rust implementation of Apache Arrow

Drop-in Apache Spark replacement written in Rust, unifying batch processing, stream processing, and compute-intensive AI workloads.

LakeSoul is an end-to-end, realtime cloud-native Lakehouse framework for fast data ingestion, concurrent updates, incremental analytics, multimodal data processing and vector search — powering next-generation BI and AI workloads.

Database connectivity API standard and libraries for Apache Arrow

Pushdown cache for DataFusion

MongoDB integrations for Apache Arrow. Export MongoDB documents to numpy array, parquet files, and pandas dataframes in one line of code.

Python module that provides a simple and convenient way to interact with InfluxDB 3.0.

cuDF - GPU DataFrame Library