Find work in Canada.

347,202 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 8 mins ago
R
Branch Manager Advisor
RBC
Wawanesa, MB
- Full-time
Lead your team from the front, actively engaging clients inside and outside the branch, providing complex advice solutions and modelling key behaviors with an emphasis on teamwork to meet…
Added 8 mins ago
R
Director Data Center Engineering
RBC
Toronto, ON
- Full-time
The role is for Global Data Center engineering lead managing a high performing team of senior engineers responsible for all the networking technology running in the RBC’s global data…
Added 8 mins ago
M
Développeur(euse) Web Fullstack
Mirego
Montreal, QC · Quebec, QC
- Full-time
As part of the development team, you contribute every day to building innovative digital products with the ambition to change the world, no matter of their scale or industry. Equally…
Added 8 mins ago
M
Contrôleur(e) financier(ère)
Mirego
Quebec, QC
- Full-time
…leveraging technology and artificial intelligence to simplify your team's work. You know your success is measured by financial information that is more reliable and available more quickly,…
Added 8 mins ago
M
Développeur(euse) IA
Mirego
Montreal, QC · Quebec, QC
- Full-time
- You lead your AI projects from solution design all the way through to production implementation, including prototyping and technical hypothesis validation.
Added 8 mins ago
M
Directeur(trice) développement des affaires
Mirego
Montreal, QC · Quebec, QC
- Full-time
…objective is to generate qualified opportunities for the sales team by deeply understanding potential clients' needs and aligning those needs with Mirego's service…
Added 8 mins ago
P
Senior Associate -Cyber Security Forward Deployed(Bilingual FR/EN)
PwC
Calgary, AB · Vancouver, BC · Ottawa, ON · Toronto, ON
- Full-time
…our clients. You will architect deeply technical solutions, lead cross-functional engineering pods, and convert client ambiguity into shipped capabilities. You will simultaneously drive…
Added 8 mins ago
P
Technology Strategy & Transformation, Manager
PwC
Calgary, AB
- Full-time
Collaborate with team members at various levels to lead and develop compelling client proposals, white papers, thought leadership and point of view, describing the business challenge and…
Added 8 mins ago
MP
Senior Mining Consultant - Underground
Mining Plus
Vancouver, BC
- Full-time
Mining Plus continues to grow across North America, and we’re looking for a Senior Mining Consultant – Underground Mining to join our Canadian team.
Added 8 mins ago
MP
Principal Mining Consultant-Underground
Mining Plus
Vancouver, BC
- Full-time
Mining Plus continues to grow across North America, and we’re looking for a Principal Mining Consultant – Underground Mining to join our Canadian team.
Added 8 mins ago
P
Manager -Financial Crime and AML
PwC
Toronto, ON · Montreal, QC
- Full-time
Lead and support Financial Crime advisory engagements from planning through delivery, including managing scope, timelines, budgets, risks and deliverable quality
Added 8 mins ago
W
Sr. Software Development Engineer / Principal, Software Development Engineer
Workday
Vancouver, BC · Canada, BC, Vancouver
- Full-time
Lead the team technically by making critical design decisions that drive performance, reliability, and scalability across the platform.
Added 8 mins ago
BR
Safety and Risk Compliance Manager
Brad Ryan - Wright Canada Holdings LTD
Prince George, BC · CAN - SRGI Office - BC
- Full-time
Lead by example in promoting a safe work environment by consistently adhering to and reinforcing the company’s safety policies, procedures, and guidelines.
Added 8 mins ago
PH
Translation Services Coordinator
Public Health Ontario
Toronto, ON
- Full-time
Under the direction of the Team Lead, Production Development and Publishing, supports the implementation and execution of PHO's French Language Services (FLS) policy and strategy as well as…
Added 8 mins ago
SN
104R2-26-3 Indigenous Transitions Facilitator
Six Nations
Ohsweken, ON
- Full-time
Added 8 mins ago
SC
shift supervisor - Store# 04965, 404 & Major Mackenzie
Starbucks Coffee Company
Markham, ON
- Full-time
…in executing store operations during scheduled shifts. As a team lead, you will deploy partners and delegate tasks to create the Starbucks Experience for our customers by providing…
Added 8 mins ago
SC
shift supervisor - Store# 04448, BLOOR & JARVIS
Starbucks Coffee Company
Toronto, ON
- Full-time
…in executing store operations during scheduled shifts. As a team lead, you will deploy partners and delegate tasks to create the Starbucks Experience for our customers by providing…
Added 8 mins ago
SC
shift supervisor - Store# 04863, FAIRVIEW MALL
Starbucks Coffee Company
Toronto, ON
- Full-time
…in executing store operations during scheduled shifts. As a team lead, you will deploy partners and delegate tasks to create the Starbucks Experience for our customers by providing…
Added 8 mins ago
SC
shift supervisor - Store# 04327, West Broadway & Granville
Starbucks Coffee Company
Vancouver, BC
- Full-time
…in executing store operations during scheduled shifts. As a team lead, you will deploy partners and delegate tasks to create the Starbucks Experience for our customers by providing…
Added 8 mins ago
SC
shift supervisor - Store# 04762, FAIRWAY PLAZA - KITCHENER
Starbucks Coffee Company
Kitchener, ON
- Full-time
…in executing store operations during scheduled shifts. As a team lead, you will deploy partners and delegate tasks to create the Starbucks Experience for our customers by providing…
Added 8 mins ago
SC
shift supervisor - Store# 50276, 959 CHEMONG ROAD
Starbucks Coffee Company
Peterborough, ON
- Full-time
…in executing store operations during scheduled shifts. As a team lead, you will deploy partners and delegate tasks to create the Starbucks Experience for our customers by providing…
Added 8 mins ago
SC
shift supervisor - Store# 57236, SHERWOOD DRIVE & EMERALD DRIVE
Starbucks Coffee Company
Sherwood Park, AB
- Full-time
…in executing store operations during scheduled shifts. As a team lead, you will deploy partners and delegate tasks to create the Starbucks Experience for our customers by providing…
Added 8 mins ago
SC
shift supervisor - Store# 80845, 265 WEBER STREET NORTH
Starbucks Coffee Company
Waterloo, ON
- Full-time
…in executing store operations during scheduled shifts. As a team lead, you will deploy partners and delegate tasks to create the Starbucks Experience for our customers by providing…
Added 8 mins ago
SC
shift supervisor - Store# 04513, The Core
Starbucks Coffee Company
Calgary, AB
- Full-time
…in executing store operations during scheduled shifts. As a team lead, you will deploy partners and delegate tasks to create the Starbucks Experience for our customers by providing…
Added 8 mins ago
SC
shift supervisor - Store# 04412, BANKERS HALL
Starbucks Coffee Company
Calgary, AB
- Full-time
…in executing store operations during scheduled shifts. As a team lead, you will deploy partners and delegate tasks to create the Starbucks Experience for our customers by providing…

15Rock

Applied AI Engineer - Evaluation & Reliability

Toronto, ON, Canada
Greater Toronto Area, Canada, Canada

Full-time

Apply About employer

Applications open in a new tab.

Job details

Job type: Full-time
Apply by: Apr 7, 2027
Posting language: English
Posted: Oct 9, 2026

Original job description

Applied AI Engineer - Evaluation & ReliabilityThe problem you'd own

15Rock is a financial intelligence firm. Our research goes in front of institutional investors and gets read line by line - by people looking for the one number that doesn't hold up. Every figure has to reconcile across every table and sentence; every claim has to trace to a live source.

We build that research with a multi-agent pipeline on large language models. LLMs write well and count badly: left alone, they produce clean prose that contradicts the table three paragraphs up. Our answer isn't "prompt harder." It's to route every number through deterministic computation and gate every report behind checks that pass or fail - never a model grading its own work. You'd build that layer.

What you'll build

Deterministic evaluation gates that block a report when a defect is present - measured against thresholds set before the run, not a gut feel.
A fact-reconciliation layer that verifies a figure agrees with itself everywhere it appears.
Citation-integrity checks: every reference resolves, every quote is real, every source is live.
Golden-set and regression harnesses that catch quality drift before it ships.
The path from research to production: hardening a validated approach into the pipeline, behind CI, with the traces to prove it works.

What we're looking for

You've shipped LLM systems to production - not demos - where being wrong had a cost.
You've built the evaluations that kept them honest: you think in precision and recall, and you define "good" before you measure it.
Strong Python; you can own backend services and data pipelines yourself.
You reason about the numbers, not just the tokens - enough quantitative or financial literacy to know when a figure is wrong, or the appetite to learn it fast.
Self-directing. You'd rather write a one-page memo than wait for a ticket.

Bonus: model-risk or evaluation-infrastructure work in a regulated setting; LangGraph, DuckDB, CI you owned; a finance or quantitative background.

This role is not

Prompt engineering, a pure research seat, or a place where "the model said so" is an answer.

How we work

A small, senior team with direct access to the founder and research lead. Async by default - we meet on blockers, not to read status aloud. We run on hypotheses and measurable outcomes.

Details

Downtown Toronto, on-site / hybrid. Must be authorized to work in Canada. Early-stage financial intelligence firm, recognized by the Globe and Mail Changemaker Award and selected for Google for Startups.

To Apply: send a short note and something you've built where correctness mattered to Careers@15Rock.com . Tell us about a check you built that caught something a person would have missed.

15Rock is an equal-opportunity employer.

Apply About employer

More about this job

Responsibilities

Build deterministic evaluation gates that block reports with defects, and develop checks for fact consistency, citation validity, and source integrity. Create golden-set and regression testing harnesses, then harden validated approaches for production with CI and traceable evidence.

Requirements

Candidates should have shipped production LLM systems where errors carried real costs and built evaluations using defined quality measures such as precision and recall. Strong Python skills and the ability to own backend services and data pipelines are required, along with quantitative or financial literacy or the ability to learn it quickly.

Skills

Python
LLM Production Systems
Evaluation Design
Precision and Recall
Deterministic Computation
Fact Reconciliation
Citation Integrity
Regression Testing
Backend Services
Data Pipelines
Quantitative Analysis
Financial Literacy
CI
LangGraph
DuckDB

Visa sponsorship

Not detected in the job text

Keywords

Applied AI
Large Language Models
Multi-Agent Pipeline
Python
Deterministic Computation
Evaluation Gates
Precision
Recall
Fact Reconciliation
Citation Integrity
Golden Sets
Regression Testing
Backend Services
Data Pipelines
Production Hardening
Continuous Integration
LangGraph
DuckDB
Model Risk
Evaluation Infrastructure
Financial Intelligence
Quantitative Analysis
Institutional Investors
Source Verification

Report this job