Responsibilities
Design, build, and maintain scalable, automated data pipelines for batch and streaming workloads using Azure data platforms, including Databricks and Delta Live Tables. Develop data architectures and models, orchestrate pipeline workflows, and monitor pipeline health and performance.
Requirements
Proficiency in Python, PyTest, and PySpark, strong software engineering practices, and knowledge of object-oriented programming are required. The role also calls for experience with Azure, Databricks, Delta Live Tables, Unity Catalog, data modeling, orchestration, monitoring, and Agile development.
