Stream Processing GitHub Repositories (2026) · Open Source Tools

Cross-platform, customizable ML solutions for live and streaming media.

A high-performance observability data pipeline.

Asynchronous Python framework for event-driven services. A thin client for Kafka, RabbitMQ, NATS, Redis and MQTT with full access to native broker features, plus AsyncAPI docs, in-memory tests and observability out of the box.

A complete Sigma detection engineering toolkit: parser, linter, formatter, evaluator, correlation engine, conversion framework, streaming daemon, MCP and LSP servers :crab:

Event streaming platform for agentic AI. Continuously ingest, transform, and serve event streams in real time, at scale.

Upserts, Deletes And Incremental Processing on Big Data.

Fast, Simple and a cost effective tool to replicate data from Postgres to Data Warehouses, Queues and Storage

Zero-cost & Type-safe Reactive Extensions for Rust. Supports both high-performance single-threaded (Local) and thread-safe (Shared) contexts. WASM & Tokio ready.

Apache StreamPipes - A self-service (Industrial) IoT toolbox to enable non-technical users to connect, analyze and explore IoT data streams.

Fast and Lightweight Logs, Metrics and Traces processor for Linux, BSD, OSX and Windows

Kubernetes-native platform to run massively parallel data/streaming jobs

The fastest unified streaming SQL Engine in a single C++ binary. ⚡ Millisecond latency. 100+ GB/s throughput. Continuously compute stateful context from streams, logs, metrics, events, and CDC.

Python Streaming DataFrames for Kafka

Build platforms that flexibly mix SQL, batch, and stream processing paradigms

Framework for building Event-Driven Microservices

head/tail for structured data - summarize/preview JSON/YAML and source code

A Go port of the Java Streams API. Type-safe and functional Go Streams processing

Unified real-time data engine

Open Source Phasor Data Concentrator

High-performance streaming SQL query engine designed for real-time data processing. Use cases include event-driven architectures, ETL pipelines, and modern data-intensive applications.