Find work in Canada.

329,548 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Save this search

Added 1 min ago
Agent, Comptes en Fiducie et Commissions (contrat d'1 an) / Trust Accounts and Commissions Agent (1 year contract)
PEAK Financial Group
Montreal, QC
Full-time
We are looking for an agent to support the Trust Accounts & Commissions team. In this role, you will contribute to daily reconciliations, advisors’ commission processing, and compliance…
Added 15 mins ago
Observability Engineer (KUBERNETES & CONTAINERS, GITOPS & AUTOMATION, QUERY LANGUAGES, CLOUD & INFRASTRUCTURE)
Apptoza Inc.
Toronto, ON
Contract
…observability data • Implement Model Context Protocol (MCP) for AI agent integration with observability platforms AUTOMATION & PLATFORM…
Added 31 mins ago
Stage : Superviseur COOP-Hiver 2027 | Internship - COOP supervisor – Winter 2027
RTX
Longueuil, QC
Full-time
…collaboration avec le superviseur (assurer la fluidité des opérations au quotidien, assigner les tâches, agir en tant que personne ressource pour le département, appliquer les règles de la…
Added 46 mins ago
Senior Software Engineer, Internal Tools
Tubi - Canada
Toronto, ON
…from developer resources to production-grade AI for business operations. We are the group responsible for turning AI from an experiment into an operating capability: training,…
Added 1 hour ago
Document Management Representative
Fidelity Canada
Toronto, ON
Full-time
…distribution of work items to all business units in the Transfer Agent Operations and…
Added 1 hour ago
Guest Service Agent
Northland Properties
Langley (District), BC · Langley, BC
First impressions count, right? We think so too! The Guest Service Agent is the face of our hotels. You're a people-pleaser who loves to go above and beyond to exceed the needs of our…
Added 1 hour ago
Guest Service Agent
Northland Properties
Edmonton, AB
First impressions count, right? We think so too! The Guest Service Agent is the face of our hotels. You're a people-pleaser who loves to go above and beyond to exceed the needs of our…
Added 2 hours ago
AI Engineer, Mid Level
Jobright.ai
Canada
Full-time
Review code, document systems, and support production operations
Added 2 hours ago
Regulated Pharmacy Technician
Walmart Canada
Port Elgin, ON
Full-time
…and the prescription is released to the correct customer or…
Added 2 hours ago
Senior AI Engineer
CI Financial
Toronto, ON
Full-time
You'll be building the agents and GenAI systems that CI's advisors, analysts, and operations teams actually rely on day to day - not prototypes that stall out after the demo.
Added 2 hours ago
Royal Service Agent - Full Time
AccorHotel
Vancouver, BC
Full-time
As a Royal Service Agent, you’ll be one of the primary points of contact for our guests throughout their stay, providing warm, personalized and efficient service by phone, text, email and…
Added 2 hours ago
Insurance Agent with a Level 1 or 2 General Insurance license - Sabil Insurance
The InsureBC Group
Aldergrove, BC
Full-time
Support day-to-day office operations and contribute positively to a professional working environment
Added 2 hours ago
Operations Excellence Project Manager
Agrocrop Exports Ltd.
Bolton, ON
Full-time
The Operations Excellence Project Manager is responsible for leading operational improvement initiatives across manufacturing, warehousing, supply chain, and support functions. The role…
Added 3 hours ago
Advisory Agent, Member and Client Experience
Desjardins
Quebec, QC · QC · Saint-Raymond, QC · Saint-Marc-des-Carrières, QC · Sainte-Catherine, QC · Ste.-Foy, QC · Caisse populaire Desjardins de Saint-Raymond-Sainte-Catherine
Full-time
Support members and clients by telling them existing applications and technological tools that can help them do transactions and operations on their own.
Added 3 hours ago
Agent, Office work and operations
Desjardins
Levis, QC
Full-time
As an office work and operations agent, you provide administrative and operational support to the organization. You strive to maximize operational productivity. This may include working in…
Added 3 hours ago
Advisory Agent, Member and Client Experience
Desjardins
Montreal, QC · Caisse Desjardins du Sud-Ouest de Montréal
Part-time
As a member and client experience advisory agent, you help give members and clients a better experience, and ensure that they are satisfied. This includes making sure they feel welcome,…
Added 3 hours ago
Senior Developer, Development Services
BDO
Toronto, ON · Oakville, ON · Markham, ON
Full-time
Drive technical decisions across the full software development lifecycle, including architecture, development, testing, deployment, and ongoing support.
Added 3 hours ago
Staff Data Scientist, AI Evaluations Platform
RBC
Toronto, ON · Montreal, QC
Full-time
…science function responsible for how RBC measures model and agent quality, safety, risk, and performance. You will design, build, and continuously improve evaluation datasets, sourcing…
Added 3 hours ago
Document Management Representative
Fidelity Canada
Toronto, ON
Full-time
…distribution of work items to all business units in the Transfer Agent Operations and…
Added 3 hours ago
Senior Integration Architect
Infotree Global Solutions
Toronto, ON
Contract
…AI/Agentic Orchestration. Drives digital transformation of Legal operations, establishes platform governance, and mentors junior…
Added 4 hours ago
Front Desk Agent - Front Office
Hard Rock Hotel & Casino Ottawa
Ottawa, ON
$20.05 CAD / hour
Part-time
The Front Desk Agent is responsible for welcoming hotel and casino guests and managing check-in and check‑out processes. This role handles reservations, billing, room assignments, and guest…
Added 4 hours ago
Advanced Agent - Guest Communications
Vail Resorts
Whistler, BC
Added 4 hours ago
Community Services Manager
House of Friendship
Waterloo, ON
Full-time
…located across Waterloo region. We are seeking a leader who can support the team to ensure our programs are community-driven and grounded in anti-racist, anti-oppressive, and…
Added 4 hours ago
Generative AI Cloud Operations Engineer - Evinova
Evinova
Mississauga, ON
Full-time
The Machine Learning and Artificial Intelligence Operations team (ML/AI Ops) is a newly formed platform team that will spearhead the design, creation, and operational excellence of our…
Added 4 hours ago
Sanajiksanut - Agent, Human Resources (Fly-In Fly-Out)
Agnico Eagle Mines Limited
Baker Lake, NU
Full-time
Want to learn more about our operations? Click here!

RBC

Staff Data Scientist, AI Evaluations Platform

Toronto, ON, Canada
Montreal, QC, Canada

Job type: Full-time
Apply by: Oct 5, 2026
Posting language: English
Posted: Sep 21, 2026

Apply About employer

Applications open in a new tab.

Original job description

Job Description

What is the opportunity?

RBC’s AI Group is building trusted AI capabilities for the enterprise, and evaluation is one of the core controls that makes that possible. As Staff Data Scientist, AI Evaluations, you will be a senior technical leader in the data science function responsible for how RBC measures model and agent quality, safety, risk, and performance. You will design, build, and continuously improve evaluation datasets, sourcing methods, LLM judge design, deterministic scorers, human evaluation protocols, quality benchmarks, and measurement frameworks that help AI systems move from experimentation to production with evidence and control, while providing technical guidance and mentorship to other data scientists on the team.

What will you do?

Design and drive advanced model and agent evaluation methodologies, including evaluation datasets, rubrics, LLM-as-judge methods, deterministic scorers, human evaluation, and measurement frameworks, providing technical guidance to junior team members.
Define evaluation science standards that translate model risk, responsible AI, product quality, safety, and business expectations into measurable criteria, repeatable methods, and clear evidence.
Own the end-to-end lifecycle for evaluation datasets and scorecards, including sourcing, curation, validation, quality checks, versioning, lineage, reuse, and ongoing improvement.
Design scalable evaluation approaches for generative AI and agentic systems, including task-level, workflow-level, trajectory-level, and runtime evaluation methods.
Partner with AI research, platform engineering, product, risk, governance, and business teams to embed evaluations into build, release, certification, monitoring, and recertification workflows.
Establish human evaluation and review protocols that produce reliable labels, reviewer guidance, adjudication processes, quality controls, and audit-ready evidence.
Measure and improve scorer accuracy, calibration, robustness, failure-mode coverage, and explainability across automated and human evaluation approaches.
Provide clear technical leadership, executive-ready communication, and mentorship to junior data scientists to help RBC scale trusted AI with speed, rigor, and control.
In this role, you will communicate and interact frequently with RBC partners and/or employees located across Canada and/or worldwide

What do you need to succeed?

Must Have

8+ years of experience in data science, applied machine learning, AI evaluation, ML quality, or a related technical field, including experience providing technical leadership and mentorship within a team.
Strong experience designing evaluation frameworks for ML, generative AI, or agentic AI systems, including metrics, datasets, benchmarks, rubrics, and quality measurement.
Practical experience with LLM evaluation methods such as LLM-as-judge, deterministic scoring, human evaluation, hallucination assessment, factuality assessment, safety evaluation, or model quality benchmarking.
Strong technical foundation in data science, statistics, machine learning, experimentation, data curation, Python, SQL, and modern AI/ML development practices.
Proven ability to translate governance, model risk, responsible AI, and business requirements into measurable controls, repeatable evaluation processes, and decision-ready evidence.
Strong communication and stakeholder management skills, with the ability to influence senior leaders across research, engineering, product, governance, risk, and business teams.

Nice to Have

Experience evaluating agentic AI systems, tool-calling workflows, multi-step reasoning, runtime traces, trajectory scoring, or workflow-level performance.
Experience in financial services, regulated AI, model risk management, responsible AI, enterprise governance, or audit-ready evidence processes.
Familiarity with tools and platforms such as MLflow, Langfuse, LangSmith, OpenTelemetry, Grafana, CI/CD pipelines, or comparable evaluation and observability tooling.
Publications, patents, open-source contributions, or industry work related to AI evaluation, ML quality, AI safety, applied research, or responsible AI.

What’s in it for you?

We thrive on the challenge to be our best, progressive thinking to keep growing, and working together to deliver trusted advice to help our clients thrive and communities prosper. We care about each other, reaching our potential, making a difference to our communities, and achieving success that is mutual.

A comprehensive Total Rewards Program including bonuses and flexible benefits, competitive compensation, commissions, and stock where applicable
Leaders who support your development through coaching and managing opportunities
Ability to make a difference and lasting impact
Work in a dynamic, collaborative, progressive, and high-performing team
A world-class training program in financial services
Opportunities to do challenging work.

#TECHPJ

Job Skills

Big Data Analytics, Critical Thinking, Decision Making, Industry Knowledge, Machine Learning (ML), Results-Oriented, Software Engineering, Software Product Design

Additional Job Details

Address:

RBC WATERPARK PLACE, 88 QUEENS QUAY W:TORONTO

City:

Toronto

Country:

Canada

Work hours/week:

37.5

Employment Type:

Full time

Platform:

TECHNOLOGY AND OPERATIONS

Job Type:

Regular

Pay Type:

Salaried

Posted Date:

2026-07-13

Application Deadline:

2026-10-05

Note: Applications will be accepted until 11:59 PM on the day prior to the application deadline date above

Our Employment Opportunities

At RBC, we are guided by living shared values of Client First, Integrity, Collaboration, Respect and Excellence and winning together as One RBC. We believe an inclusive workplace that has diverse perspectives is core to our continued growth as one of the largest and most successful banks in the world. Maintaining a workplace where our employees feel supported to perform at their best, effectively collaborate, drive innovation, and grow professionally helps to bring our Purpose to life and create value for our clients and communities. RBC strives to deliver this through policies and programs intended to foster a workplace based on respect, belonging and opportunity for all.

Join our Talent Community

Stay in-the-know about great career opportunities at RBC. Sign up and get customized info on our latest jobs, career tips and Recruitment events that matter to you.

Expand your limits and create a new future together at RBC. Find out how we use our passion and drive to enhance the well-being of our clients and communities at jobs.rbc.com.

RBC is presently inviting candidates to apply for this existing vacancy. Applying to this posting allows you to express your interest in this current career opportunity at RBC. Qualified applicants may be contacted to review their resume in more detail.

Apply About employer

More about this job

Responsibilities

You will lead the design and implementation of evaluation frameworks for AI models and agents to ensure quality, safety, and performance. This involves collaborating with cross-functional teams to embed these evaluations into the end-to-end AI development lifecycle.

Requirements

Candidates must have over 8 years of experience in data science or applied machine learning with a strong technical foundation in AI evaluation methodologies. Proven ability to translate governance and risk requirements into measurable controls and provide technical leadership is essential.

Skills

Data Science
Machine Learning
AI Evaluation
LLM Evaluation
Python
SQL
Statistics
Data Curation
Model Risk Management
Responsible AI
Stakeholder Management
Technical Leadership
Generative AI
Agentic AI
Experimentation
Governance

Benefits

Bonuses
Flexible benefits
Competitive compensation
Stock options

Visa sponsorship

Not detected in the job text

Keywords

Data Science
Machine Learning
AI Evaluation
Generative AI
Agentic AI
LLM-as-judge
Python
SQL
MLflow
Langfuse
LangSmith
OpenTelemetry
Grafana
CI/CD
Model Risk Management
Responsible AI
Governance
Data Curation
Statistics
Benchmarking
Human Evaluation
Hallucination Assessment
Factuality Assessment
Financial Services
Technical Leadership
Stakeholder Management
Experimentation
Observability
Model Quality
AI Safety

Report this job