Sr Data Engineer -
- ON, Canada
- East Toronto (Studio District), ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Posting language
- English
- Posted
Original job description
Job Summary
We are seeking a highly skilled Databricks Engineer with
AI/ML experience to design, build, and optimize scalable data and machine
learning platforms on Databricks. The role involves end-to-end ownership of
data pipelines, ML workflows, and production AI systems.
Key Responsibilities
- Design
and implement scalable ETL pipelines using Databricks & Spark
- Build
Lakehouse architecture using Delta Lake
- Develop
and deploy ML models using MLflow
- Implement
MLOps pipelines for training, testing, and serving models
- Optimize
cluster performance and reduce compute cost
- Build
RAG and LLM-based solutions using Mosaic AI
- Integrate
analytics with BI tools (Power BI, Tableau)
- Implement
data governance using Unity Catalog
- Collaborate
with Data Scientists and Business teams
- Ensure
data quality, security, and compliance
Required Skills
Mandatory
- 5+
years of Databricks & Apache Spark
- Strong
Python & PySpark
- Experience
with Delta Lake & Lakehouse
- MLflow
& MLOps experience
- Cloud
platform (AWS/Azure/GCP)
- Git
& CI/CD
Preferred
- Experience
with LLMs & Generative AI
- RAG
pipelines & Vector Databases
- Deep
Learning frameworks
- Databricks
certifications
- Power
BI integration
Requirements
Job Summary
We are seeking a highly skilled Databricks Engineer with
AI/ML experience to design, build, and optimize scalable data and machine
learning platforms on Databricks. The role involves end-to-end ownership of
data pipelines, ML workflows, and production AI systems.
Key Responsibilities
- Design
and implement scalable ETL pipelines using Databricks & Spark
- Build
Lakehouse architecture using Delta Lake
- Develop
and deploy ML models using MLflow
- Implement
MLOps pipelines for training, testing, and serving models
- Optimize
cluster performance and reduce compute cost
- Build
RAG and LLM-based solutions using Mosaic AI
- Integrate
analytics with BI tools (Power BI, Tableau)
- Implement
data governance using Unity Catalog
- Collaborate
with Data Scientists and Business teams
- Ensure
data quality, security, and compliance
Required Skills
Mandatory
- 5+
years of Databricks & Apache Spark
- Strong
Python & PySpark
- Experience
with Delta Lake & Lakehouse
- MLflow
& MLOps experience
- Cloud
platform (AWS/Azure/GCP)
- Git
& CI/CD
Preferred
- Experience
with LLMs & Generative AI
- RAG
pipelines & Vector Databases
- Deep
Learning frameworks
- Databricks
certifications
- Power
BI integration
More about this job
Responsibilities
Design, build, and optimize scalable data and machine learning platforms using Databricks and Spark. Manage end-to-end data pipelines, ML workflows, and production AI systems while ensuring data quality and security.
Requirements
Requires 5+ years of experience with Databricks and Apache Spark along with strong Python and PySpark skills. Proficiency in Delta Lake, MLOps, and cloud platforms is mandatory, with preference for experience in Generative AI and RAG pipelines.
Skills
- Databricks
- Apache Spark
- Python
- PySpark
- Delta Lake
- Lakehouse
- MLflow
- MLOps
- AWS
- Azure
- GCP
- Git
- CI/CD
- Generative AI
- RAG
- Vector Databases
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Software
- Technology
- Engineering
Keywords
- Databricks
- Apache Spark
- Python
- PySpark
- Delta Lake
- Lakehouse
- MLflow
- MLOps
- AWS
- Azure
- GCP
- Git
- CI/CD
- Generative AI
- RAG
- Vector Databases
- Deep Learning
- Mosaic AI
- Power BI
- Tableau
- Unity Catalog
- ETL
- Data Engineering
- Machine Learning
- Data Governance
- Cloud Computing