Job Title: Lead Data Engineer – Azure Databricks
Location: Burnaby, BC – Hybrid
Duration: 12 months plus
Experience: 8–10 years
RoleWe are looking for a hands-on Lead Data Engineer with strong experience in Azure Databricks and Spark to build, optimize, and support scalable data pipelines and Lakehouse solutions.Mandatory Skills4+ years hands-on Azure Databricks & Apache SparkStrong PySpark and SQLHands-on Spark performance optimization – partitioning, caching, broadcast joins, AQE, skew, shuffle and OOM troubleshootingStrong experience with Delta Lake and Unity CatalogDatabricks Jobs/Workflows, Notebooks and ReposTerraform for Databricks/Azure infrastructureADLS Gen2Azure Data Factory (ADF) and/or AirflowAzure Key VaultAzure Synapse / Azure SQLEvent Hub / Kafka and data ingestionGit and CI/CD using Azure DevOps or GitHub ActionsExperience with dev/test/prod environment managementData modeling: Lakehouse/Medallion architecture, Star SchemaFile formats: Parquet/JSONUnit testing with pytestStrong troubleshooting and implementation-level Databricks experiencePython/PySpark + SQL are mandatory; basic Scala is preferredPreferredTelecom / Supply Chain domain experienceDatabricks Data Engineer certificationAzure DP-203 certification
