akarce/e2e-structured-streaming
End-to-end data pipeline that ingests, processes, and stores data. It uses Apache Airflow to schedule scripts that fetch data from an API, sends the data to Kafka, and processes it with Spark before writing to Cassandra. The pipeline, built with Python and Apache Zookeeper, is containerized with Docker for easy deployment and scalability.
Python
Stars
23
Forks
7
Open Issues
0
Updated
7m ago
No README available
Related Projects
donnemartin/system-design-primer
Learn how to design large-scale systems. Prep for the system design interview. Includes Anki flashcards.
371.3k
Python
vinta/awesome-python
The definitive list that answers "I want to do X in Python, which tool should I use?"
322.4k
Python
practical-tutorials/project-based-learning
Curated list of project-based tutorials
284.3k
Python
TheAlgorithms/Python
All Algorithms implemented in Python
224.9k
Python
tensorflow/tensorflow
An Open Source Machine Learning Framework for Everyone
200.3k
C++
yt-dlp/yt-dlp
A feature-rich command-line audio/video downloader
193.0k
Python
Statistics
- Stars
- 23
- Forks
- 7
- Open Issues
- 0
- Created
- Jul 25, 2024
- Updated
- Sep 23, 2026