Top DATA-PIPELINE GitHub Repositories & Tools (2026)

Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

An orchestration platform for the development, production, and observation of data assets.

Open-source inference server and production cluster for all the models your agent needs.

Flink CDC is a streaming data integration tool

OLake - Fastest Databases, Kafka & S3 Replication to Apache Iceberg with Table optimization (Called OLake Fusion). ⚡ Efficient, quick and scalable data ingestion for real-time analytics. Supported sources : Postgres, MongoDB, MySQL, Oracle, MSSql, DB2, Kafka, S3.

A SQL transformation engine that type-checks your whole pipeline and catches breaking changes before they run — branches, replay, column-level lineage, compile-time contracts, per-model cost. Adapters: Databricks, Snowflake, BigQuery, DuckDB. Single static Rust binary. Apache 2.0.

Where data access meets operational intelligence