Senior / Lead Azure Data Engineer

Job Title: Senior / Lead Azure Data Engineer

Location: Toronto, ON, Canada

Experience: 10–14 Years

Employment Type: Contract / Full-Time

Domain: Banking / Financial Services

Job Summary

We are seeking an experienced Senior / Lead Azure Data Engineer with strong expertise in Microsoft Azure, Azure Data Factory, Azure Databricks, PySpark, and Azure Synapse Analytics. The ideal candidate will have hands-on experience designing and developing scalable ETL/ELT pipelines, implementing cloud-based data engineering solutions, and managing large-scale data processing workloads.

Experience working with financial or banking data, data reconciliation, data quality, and enterprise data governance will be highly valued.

Key Responsibilities

• Design, develop, and implement scalable ETL/ELT pipelines using Azure Data Factory (ADF).
• Build and optimize data processing solutions using Azure Databricks, PySpark, and Spark SQL.
• Develop data ingestion frameworks integrating multiple sources into Azure Data Lake Storage Gen2.
• Design and implement cloud-based data warehouse solutions using Azure Synapse Analytics and Azure SQL.
• Develop complex SQL queries, stored procedures, and data transformation logic.
• Implement Delta Lake architecture, Delta Live Tables (DLT), and data quality validation frameworks.
• Work with Unity Catalog to implement data governance, access controls, and data lineage.
• Develop metadata-driven and parameterized pipelines for automated data processing.
• Implement batch and near-real-time data processing solutions using Azure Event Hubs and other Azure services.
• Optimize pipeline performance, Spark jobs, SQL queries, and cloud resource utilization.
• Implement secure data access using Azure Key Vault, Azure Active Directory, and managed identities.
• Develop CI/CD processes for automated deployment across development, QA, and production environments.
• Collaborate with enterprise architects, business analysts, QA teams, and stakeholders to deliver reliable data solutions.
• Ensure data accuracy, reconciliation, exception handling, and compliance with enterprise data standards.

Mandatory Technical Skills

• Strong hands-on experience with Microsoft Azure Data Engineering.
• Azure Data Factory (ADF).
• Azure Databricks, PySpark, and Spark SQL.
• Azure Data Lake Storage Gen2 (ADLS).
• Azure Synapse Analytics.
• Strong SQL Server and Azure SQL Database experience.
• ETL/ELT pipeline development and data warehousing.
• Python programming and data transformation.
• Data integration, migration, and performance optimization.

Good to Have

• Delta Lake and Delta Live Tables (DLT).
• Unity Catalog and Databricks Asset Bundles.
• Azure Functions, Logic Apps, Event Grid, and Event Hubs.
• Azure Key Vault and Azure Active Directory.
• CI/CD using Jenkins, GitHub, or Azure DevOps.
• Banking or financial services domain experience.
• Data governance, data quality, and reconciliation.

Experience & Qualifications

• 10+ years of experience in Data Engineering / Data Warehousing.
• 5+ years of hands-on experience with Azure Data Engineering technologies.
• Experience designing and implementing enterprise-scale cloud data solutions.
• Strong understanding of Agile/Scrum and SDLC methodologies.
• Bachelor's degree in Computer Science, Engineering, or a related field.

Preferred Candidate Profile

A hands-on Senior / Lead Azure Data Engineer who can independently design, develop, optimize, and support enterprise data pipelines, with strong experience in Azure Databricks, ADF, PySpark, SQL, and cloud data architecture. Banking or financial services experience is preferred.