Senior Data Engineer – Azure & Databricks

Senior Data Engineer - Azure & Databricks

Job Summary

We are seeking a highly skilled Senior Data Engineer with strong hands-on expertise in Azure cloud data engineering and Databricks-based pipeline development. The ideal candidate will have extensive experience building scalable, high-performance data pipelines and modern data platforms using Azure-native technologies.

This is a core data engineering role focused on large-scale ETL/ELT development, orchestration, optimization, and cloud data platform engineering - not an analytics or machine learning-focused position.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT data pipelines on Microsoft Azure.

• Build and optimize data processing workflows using Databricks and PySpark.

• Develop and support data integration solutions using Azure data services including:

• ADLS

• ADF

• Synapse

• Databricks

• Implement large-scale data processing and performance optimization strategies.

• Design scalable and efficient data models to support enterprise data platforms.

• Manage data orchestration, monitoring, troubleshooting, and operational support activities.

• Support CI/CD and DevOps practices for automated deployment and data workflow management.

• Collaborate with engineering and business teams to ensure reliable and high-quality data delivery.

Required Qualifications

• Strong hands-on experience with Microsoft Azure data engineering services.

• Extensive experience with Databricks and Spark-based processing frameworks.

• Strong proficiency in PySpark and SQL development.

• Experience building end-to-end ETL/ELT pipelines and scalable cloud data solutions.

• Expertise in large-scale data processing and performance tuning.

• Strong understanding of data orchestration, monitoring, and troubleshooting practices.

• Experience implementing CI/CD and DevOps processes for data engineering workflows.

• Strong analytical and problem-solving skills.

Preferred Qualifications

• Experience working in enterprise-scale cloud migration or modernization initiatives.

• Exposure to modern lakehouse or distributed data architectures.

• Experience supporting high-volume data processing environments.

Key Skills & Competencies

• Azure Data Engineering

• Databricks & PySpark

• ETL / ELT Pipeline Development

• Azure Synapse, ADF & ADLS

• SQL & Data Modeling

• Spark Performance Optimization

• Data Orchestration & Monitoring

• CI/CD & DevOps for Data Workflows