Big Data GitHub Repositories (2026) · Open Source Tools

Apache Spark - A unified analytics engine for large-scale data processing

High-performance data engine for AI and multimodal workloads. Process images, audio, video, and structured data at any scale

A fast, scalable, high performance Gradient Boosting on Decision Trees library, used for ranking, classification, regression and other machine learning tasks for Python, R, Java, C++. Supports computation on CPU and GPU.

Apache DataFusion SQL Query Engine

Seamless multi-primary syncing database with an intuitive HTTP/JSON API, designed for reliability

Apache Auron is an accelerator for big data engines, leveraging native vectorized execution to accelerate query processing.

ClickHouse® is a real-time analytics database management system

Cloud-native OSS search engine for observability

The AI search platform

Apache DataFusion Ballista Distributed Query Engine

Apache Paimon Rust The rust implementation of Apache Paimon.

Apache IoTDB

Apache Flink Kubernetes Operator

Apache Wayang is the first cross-platform data processing system.

The official home of the Presto distributed SQL query engine for big data

Apache Hive

Official repository of Trino, the distributed SQL query engine for big data, formerly known as PrestoSQL (https://trino.io)

Apache Fluss is a streaming storage built for real-time analytics.

CrateDB is a distributed and scalable SQL database for storing and analyzing massive amounts of data in near real-time, even with complex queries. It is PostgreSQL-compatible, and based on Lucene.

World's First OSS Flink Accelerator built on Apache DataFusion