Sr Data Engineer -
- Toronto, ON, Canada
- East Toronto (Studio District), ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Posting language
- English
- Posted
Original job description
Job Summary
We are seeking a highly skilled Databricks Engineer with
AI/ML experience to design, build, and optimize scalable data and machine
learning platforms on Databricks. The role involves end-to-end ownership of
data pipelines, ML workflows, and production AI systems.
Key Responsibilities
- Design
and implement scalable ETL pipelines using Databricks & Spark
- Build
Lakehouse architecture using Delta Lake
- Develop
and deploy ML models using MLflow
- Implement
MLOps pipelines for training, testing, and serving models
- Optimize
cluster performance and reduce compute cost
- Build
RAG and LLM-based solutions using Mosaic AI
- Integrate
analytics with BI tools (Power BI, Tableau)
- Implement
data governance using Unity Catalog
- Collaborate
with Data Scientists and Business teams
- Ensure
data quality, security, and compliance
Required Skills
Mandatory
- 5+
years of Databricks & Apache Spark
- Strong
Python & PySpark
- Experience
with Delta Lake & Lakehouse
- MLflow
& MLOps experience
- Cloud
platform (AWS/Azure/GCP)
- Git
& CI/CD
Preferred
- Experience
with LLMs & Generative AI
- RAG
pipelines & Vector Databases
- Deep
Learning frameworks
- Databricks
certifications
- Power
BI integration
Requirements
Job Summary
We are seeking a highly skilled Databricks Engineer with
AI/ML experience to design, build, and optimize scalable data and machine
learning platforms on Databricks. The role involves end-to-end ownership of
data pipelines, ML workflows, and production AI systems.
Key Responsibilities
- Design
and implement scalable ETL pipelines using Databricks & Spark
- Build
Lakehouse architecture using Delta Lake
- Develop
and deploy ML models using MLflow
- Implement
MLOps pipelines for training, testing, and serving models
- Optimize
cluster performance and reduce compute cost
- Build
RAG and LLM-based solutions using Mosaic AI
- Integrate
analytics with BI tools (Power BI, Tableau)
- Implement
data governance using Unity Catalog
- Collaborate
with Data Scientists and Business teams
- Ensure
data quality, security, and compliance
Required Skills
Mandatory
- 5+
years of Databricks & Apache Spark
- Strong
Python & PySpark
- Experience
with Delta Lake & Lakehouse
- MLflow
& MLOps experience
- Cloud
platform (AWS/Azure/GCP)
- Git
& CI/CD
Preferred
- Experience
with LLMs & Generative AI
- RAG
pipelines & Vector Databases
- Deep
Learning frameworks
- Databricks
certifications
- Power
BI integration
More about this job
Responsibilities
Design and optimize scalable data and machine learning platforms using Databricks and Spark. This includes owning end-to-end data pipelines, ML workflows, and implementing RAG and LLM-based solutions.
Requirements
Requires over 5 years of experience with Databricks, Apache Spark, and Python. Candidates should be proficient in Delta Lake, MLOps, and cloud platforms, with a preference for experience in Generative AI.
Skills
- Databricks
- Apache Spark
- Python
- PySpark
- Delta Lake
- MLflow
- MLOps
- AWS
- Azure
- GCP
- Git
- CI/CD
- LLMs
- Generative AI
- RAG
- Vector Databases
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Engineering
- Consulting
Keywords
- Databricks
- Apache Spark
- Python
- PySpark
- Delta Lake
- Lakehouse
- MLflow
- MLOps
- AWS
- Azure
- GCP
- Git
- CI/CD
- LLMs
- Generative AI
- RAG
- Vector Databases
- Deep Learning
- Power BI
- Tableau
- Mosaic AI
- Unity Catalog
- ETL
- Data Governance
- Data Engineering
- AI
- Machine Learning