Find work in Canada.

344,137 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 1 min ago
A
Multimedia Design Manager
AECOM
Markham, ON
- $96,000 to $130,000 CAD / year
- Full-time
Reporting to the Marketing Director/Strategic Marketing Lead – Canada Region, this role oversees the planning, creation, and execution of multimedia, graphic design, and digital content…
Added 1 min ago
R
Principal Product Manager
REDCAN
Remote, Canada
- $150,000 to $190,000 CAD / year
- Full-time
- Remote
We are seeking a Principal Product Manager to lead product strategy and execution across multiple products and initiatives. You will own customer problems and product outcomes from…
Added 1 min ago
AC
Partner, Product Security
Air Canada
Toronto, ON · Dorval, QC
- Full-time
Lead security engagement across the full product lifecycle, from ideation through production
Added 1 min ago
Q
Technical Lead – Treasury
Quantiphi
Toronto, ON · CA ON Toronto
- Full-time
We are looking for a Technical Lead – Treasury to support a high-priority Aegis WebSeries (WS1) remediation program covering payment monitoring, access governance, and control delivery…
Added 14 mins ago
TP
Junior Coordinator, Mass Marketing
The Princess Margaret Cancer Foundation
Toronto, ON
- Internship
Help coordinate vendor and agency deliverables, estimates, timelines and production details, escalating risks or delays to the appropriate portfolio lead.
Added 14 mins ago
CC
IT Support Technician
Coca-Cola Canada Bottling Limited
Montreal, QC
- Full-time
Lead and assist OT network systems and IT operations at the production facility, ensuring timely resolution of technical issues and continuous improvement of operational performance.
Added 14 mins ago
AI
VP, IT Infrastructure & Compliance
Avant Imaging & Information Management, Inc.
Toronto, ON · Scarborough, ON
- Full-time
…clients depend on secure handling of information, reliable production systems and consistent service…
Added 14 mins ago
A
Multimedia Design Manager
AECOM
Markham, ON
- $96,000 to $130,000 CAD / year
- Full-time
Reporting to the Marketing Director/Strategic Marketing Lead – Canada Region, this role oversees the planning, creation, and execution of multimedia, graphic design, and digital content…
Added 14 mins ago
W
Quantity Surveyor
Webuild
Toronto, ON
- Full-time
Track production quantities on site and ensure alignment with payment applications and cost control records.
Added 16 mins ago
W
Production Supervisor, Siding & Trim (Woodbridge, ON)
Westlake
Woodbridge, ON · EXT - Woodbridge Siding
- Full-time
The Production Supervisor will be responsible for supervising all production and compounding efforts during assigned shift to ensure we meet Operational Key Performance Indicators (KPIs)…
Added 16 mins ago
PH
Assistant Manager, Laboratory Operations
Public Health Ontario
Peterborough, ON
- Full-time
Accountable for maintaining production and quality targets and ensuring optimal staff levels to achieve production, quality and financial targets.
Added 16 mins ago
PH
Assistant Manager, Laboratory Operations
Public Health Ontario
Orillia, ON
- Full-time
Accountable for maintaining production and quality targets and ensuring optimal staff levels to achieve production, quality and financial targets.
Added 16 mins ago
Shipper/Receiver
Nucor Rebar Fabrication
Wetaskiwin, AB
The role of Shipping/Receiving is to actively participate as a member of the division's production team, to ensure the safe and efficient loading and unloading of materials of inbound and…
Added 16 mins ago
T
Operations Analyst II
TD
Mississauga, ON
- Full-time
Coordinate with partners on key initiatives and may act as a project lead/subject matter expert for small-scale projects/initiatives in accordance with project management methodologies
Added 16 mins ago
SR
Manager, Operations - Film & TV - Maintenance
Sunbelt Rentals, Inc.
Toronto, ON
- Full-time
…regulatory compliance, quality standards, and support for Film & TV production…
Added 31 mins ago
JI
Tree Planting Staff Summer 2027
J.D. Irving, Limited
Chipman, NB
- Full-time
Earn What You're Worth - Salaries range from $230-$400/day based on experience, plus production bonuses!
Added 31 mins ago
C
Senior Python Engineer
Citi
Mississauga, ON
- Full-time
Lead hands-on implementation and architecture of Python frameworks for scalable, resilient, and performant applications.
Added 31 mins ago
C
Director Advanced Analytics and AI
CIBC
Toronto, ON
- Full-time
Lead delivery excellence - Own the end-to-end delivery of the Operations AI portfolio, from intake and prioritization through development, production, adoption, and realized value. Create…
Added 31 mins ago
A
Market Fundamentals Analyst (18-mos Fixed-Term)
AESO
Calgary, AB · KPMG Tower
- Full-time
You thrive in a dynamic, evolving environment and are energized by opportunities to lead and contribute to transformative change.
Added 31 mins ago
A
Market Fundamentals Analyst
AESO
Calgary, AB · KPMG Tower
- Full-time
Lead the development of components of the Long-Term Outlook, market assessments, and ad hoc analyses using production cost, resource adequacy, and market simulation models.
Added 45 mins ago
S
Site Supervisor
Sublatus
Canada
- Full-time
…with environmental, regulatory, and quality standards. You lead crews, coordinate equipment and subcontractors, and act as the on-site link between field teams, Project Managers, and…
Added 45 mins ago
CL
Finance & ERP Manager
Caledon Laboratories Ltd
Georgetown, ON
- $110,000 to $125,000 CAD / year
- Full-time
…educational institutions, pharmaceutical organizations, and production facilities the world…
Added 45 mins ago
S
Media Sales Strategist – Toronto, ON
Stingray
Toronto, ON
- Full-time
Own the end-to-end sales process, from lead generation and prospecting to performing client needs analysis, presenting solutions, negotiating and execution of closing strategies;
Added 45 mins ago
NC
Media Sales Strategist – Toronto, ON
Newfoundland Capital Corporation Limited
Toronto, ON
- Full-time
Own the end-to-end sales process, from lead generation and prospecting to performing client needs analysis, presenting solutions, negotiating and execution of closing strategies;
Added 46 mins ago
LC
Parts Picker - Day Shift
Linamar Corporation
Salford, ON
- Full-time
The Parts Picker is responsible for locating, selecting, and preparing parts for shipment or use in production. Reporting to the Production Supervisor, the role will be required to follow…

15Rock

Applied AI Engineer - Evaluation & Reliability

Toronto, ON, Canada
Greater Toronto Area, Canada, Canada

Full-time

Apply About employer

Applications open in a new tab.

Job details

Job type: Full-time
Apply by: Apr 7, 2027
Posting language: English
Posted: Oct 9, 2026

Original job description

Applied AI Engineer - Evaluation & ReliabilityThe problem you'd own

15Rock is a financial intelligence firm. Our research goes in front of institutional investors and gets read line by line - by people looking for the one number that doesn't hold up. Every figure has to reconcile across every table and sentence; every claim has to trace to a live source.

We build that research with a multi-agent pipeline on large language models. LLMs write well and count badly: left alone, they produce clean prose that contradicts the table three paragraphs up. Our answer isn't "prompt harder." It's to route every number through deterministic computation and gate every report behind checks that pass or fail - never a model grading its own work. You'd build that layer.

What you'll build

Deterministic evaluation gates that block a report when a defect is present - measured against thresholds set before the run, not a gut feel.
A fact-reconciliation layer that verifies a figure agrees with itself everywhere it appears.
Citation-integrity checks: every reference resolves, every quote is real, every source is live.
Golden-set and regression harnesses that catch quality drift before it ships.
The path from research to production: hardening a validated approach into the pipeline, behind CI, with the traces to prove it works.

What we're looking for

You've shipped LLM systems to production - not demos - where being wrong had a cost.
You've built the evaluations that kept them honest: you think in precision and recall, and you define "good" before you measure it.
Strong Python; you can own backend services and data pipelines yourself.
You reason about the numbers, not just the tokens - enough quantitative or financial literacy to know when a figure is wrong, or the appetite to learn it fast.
Self-directing. You'd rather write a one-page memo than wait for a ticket.

Bonus: model-risk or evaluation-infrastructure work in a regulated setting; LangGraph, DuckDB, CI you owned; a finance or quantitative background.

This role is not

Prompt engineering, a pure research seat, or a place where "the model said so" is an answer.

How we work

A small, senior team with direct access to the founder and research lead. Async by default - we meet on blockers, not to read status aloud. We run on hypotheses and measurable outcomes.

Details

Downtown Toronto, on-site / hybrid. Must be authorized to work in Canada. Early-stage financial intelligence firm, recognized by the Globe and Mail Changemaker Award and selected for Google for Startups.

To Apply: send a short note and something you've built where correctness mattered to Careers@15Rock.com . Tell us about a check you built that caught something a person would have missed.

15Rock is an equal-opportunity employer.

Apply About employer

More about this job

Responsibilities

Build deterministic evaluation gates that block reports with defects, and develop checks for fact consistency, citation validity, and source integrity. Create golden-set and regression testing harnesses, then harden validated approaches for production with CI and traceable evidence.

Requirements

Candidates should have shipped production LLM systems where errors carried real costs and built evaluations using defined quality measures such as precision and recall. Strong Python skills and the ability to own backend services and data pipelines are required, along with quantitative or financial literacy or the ability to learn it quickly.

Skills

Python
LLM Production Systems
Evaluation Design
Precision and Recall
Deterministic Computation
Fact Reconciliation
Citation Integrity
Regression Testing
Backend Services
Data Pipelines
Quantitative Analysis
Financial Literacy
CI
LangGraph
DuckDB

Visa sponsorship

Not detected in the job text

Keywords

Applied AI
Large Language Models
Multi-Agent Pipeline
Python
Deterministic Computation
Evaluation Gates
Precision
Recall
Fact Reconciliation
Citation Integrity
Golden Sets
Regression Testing
Backend Services
Data Pipelines
Production Hardening
Continuous Integration
LangGraph
DuckDB
Model Risk
Evaluation Infrastructure
Financial Intelligence
Quantitative Analysis
Institutional Investors
Source Verification

Report this job