Databricks Lead Developer
- Mississauga, ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Experience
- Experience: 8+ Years
- Posting language
- English
- Posted
Original job description
Databricks Lead Developer
Location: Mississauga, Ontario, Canada
Employment Type: Permanent
Experience: 8+ Years
Job Summary
We are seeking an experienced Databricks Lead Developer to design, develop, and optimize scalable data engineering solutions using Azure Databricks. The ideal candidate will have strong hands-on expertise in PySpark, Python, SQL, and Azure data services, along with technical leadership experience.
Key Responsibilities
- Design and develop scalable ETL/ELT pipelines using Azure Databricks, PySpark, and Python.
- Implement Delta Lake, Delta Live Tables (DLT), and Unity Catalog.
- Build and optimize data pipelines using Azure Data Factory and ADLS Gen2.
- Implement CI/CD pipelines using Azure DevOps and Git.
- Optimize data processing performance, scalability, and cost.
- Ensure data quality, governance, security, and operational excellence.
- Mentor developers and collaborate with cross-functional teams.
Mandatory Skills
- Azure Databricks, PySpark, Apache Spark, Python, and SQL
- Azure Data Factory (ADF) and ADLS Gen2
- Delta Lake, Delta Live Tables (DLT), and Unity Catalog
- Databricks Workflows
- CI/CD using Azure DevOps and Git
- Data Engineering, ETL/ELT, and Data Warehousing
Good-to-Have Skills
Scala, Azure Synapse Analytics, Snowflake, Airflow, Databricks Asset Bundles, Auto Loader, Terraform, Medallion Architecture, Data Governance, and GenAI/RAG integrations.
More about this job
Responsibilities
Design, develop, and optimize scalable data engineering pipelines and solutions using Azure Databricks, PySpark, Python, and Azure data services. Implement data governance and quality practices, improve performance and cost efficiency, build CI/CD pipelines, and mentor developers while collaborating across teams.
Requirements
Requires 8+ years of experience and hands-on expertise in Azure Databricks, PySpark, Apache Spark, Python, SQL, Azure Data Factory, and ADLS Gen2. Candidates should also have experience with Delta Lake, Delta Live Tables, Unity Catalog, Databricks Workflows, Azure DevOps, Git, ETL/ELT, and data warehousing; technical leadership experience is expected.
Skills
- Azure Databricks
- PySpark
- Apache Spark
- Python
- SQL
- Azure Data Factory
- ADLS Gen2
- Delta Lake
- Delta Live Tables
- Unity Catalog
- Databricks Workflows
- Azure DevOps
- Git
- ETL/ELT
- Data Engineering
- Data Warehousing
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
- Management & Leadership
Keywords
- Azure Databricks
- PySpark
- Apache Spark
- Python
- SQL
- Azure Data Factory
- ADLS Gen2
- Delta Lake
- Delta Live Tables
- Unity Catalog
- Databricks Workflows
- Azure DevOps
- Git
- ETL/ELT
- Data Engineering
- Data Warehousing
- CI/CD
- Scala
- Azure Synapse Analytics
- Snowflake
- Airflow
- Databricks Asset Bundles
- Auto Loader
- Terraform
- Medallion Architecture
- Data Governance
- GenAI
- RAG
- Data Quality
- Technical Leadership