Jobs in Canada

Group jobs by employerEmail me new jobs Save this search

Added 1 month ago
AI
Embedded Software Developer
Aversan Inc.
Montreal, QC
- Full-time
We are currently seeking a qualified Embedded Software Developer to join our team. In this role, you will collaborate with a multidisciplinary team to develop and maintain embedded flight…
Added 1 month ago
I
Triton Compiler Engineering Manager
Intel
Toronto, ON
- Full-time
You must possess the below minimum qualifications to be initially considered for this position
Added 1 month ago
S
Head of Data Systems (Google Stack)
Stacktics
Toronto, ON
- Full-time
Stacktics will rely on the Head of Data Systems with ownership of the Data Systems function including mentoring and support for our team of engineers, developers, marketing analysts and…
Added 1 month ago
Senior Manager, Analytics and BI
Scotiabank
Toronto, ON
…for financial services, and the ability to work effectively with cross-functional partners while building collaborative, results-driven…
Added 1 month ago
Manager, Data Initiatives - Ottawa, ON
Scotiabank
Ottawa, ON
3 + years of experience with data management tools and various query and processing languages (SQL, Python, etc.)
Added 1 month ago
Manager, Data Initiatives - Toronto, ON
Scotiabank
Toronto, ON
3 + years of experience with data management tools and various query and processing languages (SQL, Python, etc.)
Added 1 month ago
Agentic AI Architect, Kitchener, ON ( Hybrid)
Siemens
Kitchener, ON
…ever-changing challenges, customer needs, and questions from colleagues with clever concepts and creativity. We embrace change and work with curious minds re-inventing the future of work.…
Added 1 month ago
CL
Senior Cloud DevOps Engineer
Charger Logistics Inc
Brampton, ON
- Full-time
Strong Python programming with PowerShell/Bash experience.
Added 1 month ago
Associate Technical Services Engineer II
MongoDB
Toronto, ON
This role is based in Toronto and follows a hybrid working model, with a minimum expectation of three days per week in the office
Added 1 month ago
B
Senior Staff Developer
Benevity
Remote, Canada
- $137,800 to $179,080 CAD / year
- Full-time
- Remote
As a DevEX Senior Staff Developer at Benevity, you will lead Developer Experience and SDLC modernization across our technology organization. Sitting at the intersection of hands-on platform…
Added 1 month ago
B
Senior Staff Developer
Benevity
Remote, Canada
- Full-time
- Remote
As a DevEX Senior Staff Developer at Benevity, you will lead Developer Experience and SDLC modernization across our technology organization. Sitting at the intersection of hands-on platform…
Added 1 month ago
S
Software Engineer, Streaming Platform
Sentry
Remote, Canada
- $163,000 to $200,000 CAD / year
- Full-time
- Remote
This role is based in Toronto, with three days per week in the office to work closely with engineering teams
Added 1 month ago
RB
Director, DevOps Regulatory Engineering
Royal Bank of Canada
Toronto, ON
- Full-time
Strong engineering background in CI/CD, DevOps tooling and cloud platforms, with hands-on experience in Java and Python
Added 1 month ago
R
Staff Data Scientist, AI Evaluations Platform
RBC
Montreal, QC
- Full-time
…that help AI systems move from experimentation to production with evidence and control, while providing technical guidance and mentorship to other data scientists on the…
Added 1 month ago
CL
Senior Cloud DevOps Engineer
Charger Logistics Inc.
Brampton, ON
- Full-time
Strong Python programming with PowerShell/Bash experience.
Added 1 month ago
MW
Firmware Developer III
Mueller Water Products
Toronto, ON
- Full-time
Our purpose is to connect communities to water, life’s most essential resource, with exceptional, people solutions and products. Behind that purpose is a dedicated team, united by our…
Added 1 month ago
Manager, Analytics & Process Innovation
MCAP
Toronto, ON
…develop practical technical and analytical solutions using SQL, Python, machine learning, and data integration technologies. The role is accountable for implementing and continuously…
Added 1 month ago
IG
Machine Learning Engineer
IMCS Group
ON · Ontario, Canada
- Contract
Work Model: Hybrid – 2 days per week in-person at Toronto office preferred
Added 1 month ago
S
Application Developer
Systematix
Toronto, ON
- Contract
This role is Hybrid, 2 days per week onsite in downtown Toronto · Remote Work: 2 days per week in the office
Added 1 month ago
R
Staff Data Scientist, AI Evaluations Platform- EN
RBC
Montreal, QC
- Full-time
…that help AI systems move from experimentation to production with evidence and control, while providing technical guidance and mentorship to other data scientists on the…
Added 1 month ago
IW
Business Systems Analyst & Developer - Data, AI & Agentic Automation
Innotech Windows + Doors
Langley (District), BC · Langley, BC
- Full-time
The Business Systems Analyst & Developer turns business needs into practical data, automation, and AI solutions, and builds them to production standards. The role combines business…
Added 1 month ago
Senior Manager, Data Initiatives
Scotiabank
Toronto, ON
5 + years of experience with data management tools and various query and processing languages (SQL, Python, ETL)
Added 1 month ago
SL
Principal Software Engineer, MarTech Platforms & Digital Enablement
Sun Life
Toronto, ON · Waterloo, ON · Sun Life Toronto Atria
- Full-time
Strong hands-on experience with: Java / Spring Boot / Python JavaScript / TypeScript APIs and Microservices Docker and Kubernetes
Added 1 month ago
Data Scientist
Upshop
Mississauga, ON
…optimization models. In this role, model development goes hand-in-hand with operational reliability: success is measured by accurate forecasts and stable, low-latency, deterministic…
Added 1 month ago
Specialist Systems, IT
City of Mississauga
Mississauga, ON
The role follows a hybrid work model, requiring employees to be onsite in the office 3 days per week and allowing remote work for the remaining days

RBC

Staff Data Scientist, AI Evaluations Platform- EN

Montreal, QC, Canada

Full-time

This date has passed. Check with the employer before applying.

Apply on LinkedIn About employer

Applications open in a new tab.

Job details

Job type: Full-time
Listed closing date: Oct 31, 2026
Posting language: English
Posted: Oct 9, 2026

Original job description

Job Description

What is the opportunity?

RBC’s AI Group is building trusted AI capabilities for the enterprise, and evaluation is one of the core controls that makes that possible. As Staff Data Scientist, AI Evaluations, you will be a senior technical leader in the data science function responsible for how RBC measures model and agent quality, safety, risk, and performance. You will design, build, and continuously improve evaluation datasets, sourcing methods, LLM judge design, deterministic scorers, human evaluation protocols, quality benchmarks, and measurement frameworks that help AI systems move from experimentation to production with evidence and control, while providing technical guidance and mentorship to other data scientists on the team.

What will you do?

Design and drive advanced model and agent evaluation methodologies, including evaluation datasets, rubrics, LLM-as-judge methods, deterministic scorers, human evaluation, and measurement frameworks, providing technical guidance to junior team members.
Define evaluation science standards that translate model risk, responsible AI, product quality, safety, and business expectations into measurable criteria, repeatable methods, and clear evidence.
Own the end-to-end lifecycle for evaluation datasets and scorecards, including sourcing, curation, validation, quality checks, versioning, lineage, reuse, and ongoing improvement.
Design scalable evaluation approaches for generative AI and agentic systems, including task-level, workflow-level, trajectory-level, and runtime evaluation methods.
Partner with AI research, platform engineering, product, risk, governance, and business teams to embed evaluations into build, release, certification, monitoring, and recertification workflows.
Establish human evaluation and review protocols that produce reliable labels, reviewer guidance, adjudication processes, quality controls, and audit-ready evidence.
Measure and improve scorer accuracy, calibration, robustness, failure-mode coverage, and explainability across automated and human evaluation approaches.
Provide clear technical leadership, executive-ready communication, and mentorship to junior data scientists to help RBC scale trusted AI with speed, rigor, and control.
In this role, you will communicate and interact frequently with RBC partners and/or employees located across Canada and/or worldwide

What do you need to succeed?

Must Have

8+ years of experience in data science, applied machine learning, AI evaluation, ML quality, or a related technical field, including experience providing technical leadership and mentorship within a team.
Strong experience designing evaluation frameworks for ML, generative AI, or agentic AI systems, including metrics, datasets, benchmarks, rubrics, and quality measurement.
Practical experience with LLM evaluation methods such as LLM-as-judge, deterministic scoring, human evaluation, hallucination assessment, factuality assessment, safety evaluation, or model quality benchmarking.
Strong technical foundation in data science, statistics, machine learning, experimentation, data curation, Python, SQL, and modern AI/ML development practices.
Proven ability to translate governance, model risk, responsible AI, and business requirements into measurable controls, repeatable evaluation processes, and decision-ready evidence.
Strong communication and stakeholder management skills, with the ability to influence senior leaders across research, engineering, product, governance, risk, and business teams.

Nice to Have

Experience evaluating agentic AI systems, tool-calling workflows, multi-step reasoning, runtime traces, trajectory scoring, or workflow-level performance.
Experience in financial services, regulated AI, model risk management, responsible AI, enterprise governance, or audit-ready evidence processes.
Familiarity with tools and platforms such as MLflow, Langfuse, LangSmith, OpenTelemetry, Grafana, CI/CD pipelines, or comparable evaluation and observability tooling.
Publications, patents, open-source contributions, or industry work related to AI evaluation, ML quality, AI safety, applied research, or responsible AI.
Ph.D. or Master’s degree in a quantitative or technical field, such as Applied Mathematics, Engineering, Computer Science or a related computational discipline.

What’s in it for you?

We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.

A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable
Leaders who support your development through coaching and managing opportunities
Ability to make a difference and lasting impact
Work in a dynamic, collaborative, progressive, and high-performing team
A world-class training program in financial services
Opportunities to do challenging work.

#TECHPJ

Job Skills

Big Data Analytics, Critical Thinking, Decision Making, Industry Knowledge, Machine Learning (ML), Results-Oriented, Software Engineering, Software Product Design

Additional Job Details

Address:

6666 RUE ST URBAIN, FLOOR 3:MONTRÉAL

City:

Montréal

Country:

Canada

Work hours/week:

37.5

Employment Type:

Full time

Platform:

TECHNOLOGY AND OPERATIONS

Job Type:

Regular

Pay Type:

Salaried

Posted Date:

2026-10-09

Application Deadline:

2026-10-31

Note: Applications will be accepted until 11:59 PM on the day prior to the application deadline date above

Our Employment Opportunities

At RBC, we are guided by living shared values of Client First, Integrity, Collaboration, Respect and Excellence and winning together as One RBC. We believe an inclusive workplace that has diverse perspectives is core to our continued growth as one of the largest and most successful banks in the world. Maintaining a workplace where our employees feel supported to perform at their best, effectively collaborate, drive innovation, and grow professionally helps to bring our Purpose to life and create value for our clients and communities. RBC strives to deliver this through policies and programs intended to foster a workplace based on respect, belonging and opportunity for all.

Join our Talent Community

Stay in-the-know about great career opportunities at RBC. Sign up and get customized info on our latest jobs, career tips and Recruitment events that matter to you.

Expand your limits and create a new future together at RBC. Find out how we use our passion and drive to enhance the well-being of our clients and communities at jobs.rbc.com</a/span>

RBC is presently inviting candidates to apply for this existing vacancy. Applying to this posting allows you to express your interest in this current career opportunity at RBC. Qualified applicants may be contacted to review their resume in more detail.

Apply on LinkedIn About employer

More about this job

Responsibilities

Lead the design and continuous improvement of evaluation methods, datasets, benchmarks, scoring systems, and human review protocols for AI models and agentic systems. Partner across research, engineering, product, risk, governance, and business teams to embed reliable, auditable evaluations into AI development, release, monitoring, and recertification, while mentoring data scientists.

Requirements

Requires 8+ years of relevant experience, including technical leadership and mentorship, and strong expertise in evaluation frameworks for machine learning, generative AI, or agentic AI. Candidates should have a strong foundation in data science, statistics, machine learning, experimentation, data curation, Python, and SQL, plus the ability to translate governance, risk, responsible AI, and business needs into measurable controls and evidence.

Skills

Data Science
AI Evaluation
Machine Learning
Generative AI
Agentic AI
Evaluation Frameworks
LLM-as-Judge
Deterministic Scoring
Human Evaluation
Statistics
Python
SQL
Data Curation
Model Risk Management
Responsible AI
Technical Leadership

Benefits

Bonuses
Flexible Benefits
Competitive Compensation
Commissions
Stock
Coaching and Development Opportunities
Training Program

Visa sponsorship

Not detected in the job text

Keywords

Data Science
Artificial Intelligence
AI Evaluation
Machine Learning
Generative AI
Agentic AI
LLM-as-Judge
Deterministic Scoring
Human Evaluation
Evaluation Datasets
Evaluation Frameworks
Quality Benchmarks
Model Risk Management
Responsible AI
Data Curation
Statistics
Experimentation
Python
SQL
MLflow
Langfuse
LangSmith
OpenTelemetry
Grafana
CI/CD
Model Safety
Hallucination Assessment
Factuality Assessment
Audit-Ready Evidence
Financial Services

Report this job

Work requirements

Jobs in Canada