DataForge — Enterprise Data Pipeline Orchestrator
Data pipeline orchestration platform supporting dataset ingestion, Spark-based ETL processing, asynchronous job execution, and metadata management.
- Designed reusable Spark transformation workflows for cleaning, validation, aggregation, and schema enforcement of uploaded datasets.
- Integrated MinIO for object storage and PostgreSQL for metadata and job tracking.
- Implemented Docker-based service orchestration for reproducible deployment across local and cloud environments.