Senior Data Engineer – Databricks & Microsoft Fabric
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
Senior Data Engineer - Databricks & Microsoft Fabric
Work Location: Toronto, Canada
Job Summary
We are seeking a Senior Data Engineer with strong expertise in Azure Databricks and modern cloud data platforms to design, develop, and optimize scalable data pipelines, lakehouse solutions, and analytics platforms. The role focuses primarily on Databricks data engineering, with additional experience in Microsoft Fabric and Databricks Genie to support self-service analytics, conversational AI, and enterprise reporting.
Key Responsibilities
•
Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks, PySpark, Spark SQL, and Delta Lake.
•
Implement Medallion Architecture (Bronze, Silver, Gold) and integrate data from APIs, databases, files, ERP, CRM, and cloud storage.
•
Optimize Spark jobs, cluster configurations, and data processing workloads for performance and cost efficiency.
•
Develop reusable frameworks for data ingestion, transformation, and quality validation.
•
Implement CI/CD pipelines using Azure DevOps or Git-based deployment practices.
•
Monitor and troubleshoot production data pipelines.
•
Develop data solutions using Microsoft Fabric components, including Data Factory, Lakehouse, Warehouse, Notebooks, and Data Pipelines.
•
Support data modernization initiatives, Power BI semantic models, and enterprise reporting solutions.
•
Support Databricks Genie implementation, semantic layer development, and natural language querying for self-service analytics.
•
Apply data governance, security, and access controls across data platforms.
•
Collaborate with business analysts, architects, and stakeholders to translate business requirements into technical solutions.
Required Qualifications
•
Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field.
•
7+ years of experience in data engineering.
•
4+ years of hands-on experience with Azure Databricks.
•
Strong proficiency in PySpark, Spark SQL, Python, SQL, and Delta Lake.
•
Experience with Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), and Git/Azure DevOps.
•
Strong understanding of ETL/ELT development, data modeling, data warehousing, and performance optimization.
•
Experience designing and delivering enterprise cloud data platforms.
•
Experience developing scalable data pipelines and implementing data quality practices.
Nice to Have
•
Experience with Microsoft Fabric, including Lakehouse, Warehouse, Data Factory, Notebooks, and Data Pipelines.
•
Familiarity with Power BI and semantic model development.
•
Experience with Databricks Genie and AI-powered conversational analytics.
•
Knowledge of Unity Catalog, Databricks Workflows, REST APIs, and data governance.
•
Experience with AI/GenAI-enabled analytics solutions.
•
Databricks Associate or Professional certification.
More about this job
Responsibilities
The Senior Data Engineer at Querentia Inc. will build and maintain data pipelines and lakehouse platforms, tune Spark workloads, and support reporting and self-service analytics. The work also includes deploying pipelines with CI/CD, monitoring production data, and working with business teams on requirements.
Requirements
Candidates need a bachelor's degree in a technical field, 7+ years in data engineering including 4+ years hands-on with Azure Databricks, and strong PySpark, Spark SQL, Python, SQL and Delta Lake skills. Experience with Azure Data Lake Storage, Azure Data Factory, Git or Azure DevOps and enterprise cloud data platforms is also required.
Skills
- Azure Databricks
- PySpark
- Spark SQL
- Python
- SQL
- Delta Lake
- Azure Data Lake Storage
- Azure Data Factory
- Git
- Azure DevOps
- CI/CD
- Microsoft Fabric
- Data Factory
- Lakehouse
- Data Warehouse
- Notebooks
- Data Pipelines
- Power BI
- Semantic models
- Databricks Genie
- Unity Catalog
- Databricks Workflows
- REST APIs
- Medallion Architecture
- ETL/ELT
Categories
- Technology
- Data & Analytics
- Software
Keywords
- Azure Databricks
- PySpark
- Spark SQL
- Delta Lake
- Microsoft Fabric
- Databricks Genie
- Azure Data Factory
- Azure Data Lake Storage
- Azure DevOps
- Power BI
- Unity Catalog
- Databricks Workflows
- data engineering
- ETL
- ELT
- data lakehouse
- Medallion Architecture
- data warehousing
- data pipeline
- data governance
Seniority
Senior
Tools
- Azure Databricks
- Delta Lake
- Azure Data Factory
- Azure Data Lake Storage
- Azure DevOps
- Microsoft Fabric
- Power BI
- Databricks Genie
- Unity Catalog
- Databricks Workflows
Languages
- English
Fields of study
- Computer Science
- Engineering
- Information Technology