Find work in Canada.

344,748 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 12 mins ago
AL
Civil Engineer (Geotechnical)
AGAT Laboratories
Calgary, AB
- Full-time
…and rock testing procedures. We are currently seeking a Civil Engineer with a P.Eng. designation, specializing in Geotechnical Engineering, to support our laboratory-based geotechnical…
Added 12 mins ago
CM
Propulsion Engineer
Carter Murray
Montreal, QC
- $75,000 to $150,000 CAD / year
- Full-time
Our client is seeking a Propulsion Engineer to support commercial aircraft engine maintenance, repair, and overhaul (MRO) operations, with a primary focus on CFM56 engines. This role is…
Added 13 mins ago
4th Class Power Engineer
Mondi AG
Hinton, AB
Operate and monitor steam plant equipment including boilers, turbines, and auxiliary systems.
Added 13 mins ago
Control and Automation Engineer
Hatch
Fredericton, NB
…best-in-class team leverages methodologies, governance and systems that are unparalleled in the engineering space. Are you looking for an opportunity to join a diverse group of…
Added 13 mins ago
C
Software Engineer
CARFAX
London, ON
- $78,000 to $116,000 CAD / year
- Full-time
Apply best practices for architecture, security, and performance of complex systems primarily in Typescript.
Added 13 mins ago
C
Senior Software Engineer
CARFAX
London, ON
- $93,000 to $139,000 CAD / year
- Full-time
Apply best practices for architecture, security, and performance of complex systems primarily in Typescript.
Added 13 mins ago
W
Ingénieur en test et conception mécanique / Test & Mechanical Engineer
Wabtec
Montreal, QC
- Full-time
Added 13 mins ago
Stationary Engineer
Arauco
Sault Ste. Marie, ON
Responsible for the Heat Plant operation and operating maintenance reporting to the Chief Operating Engineer.
Added 27 mins ago
A
Staff Software Engineer - Open Source Data Platform (ODP)
Acceldata
Kitchener, ON
- Full-time
We are seeking a Staff Engineer with at least 9 years of experience in distributed systems or data platform development. In this role, you will be a technical leader responsible for…
Added 27 mins ago
S
Artificial Intelligence Consultant
SMARTFOX
Canada
- Full-time
Work with data engineers and developers to connect solutions to client systems, data pipelines, and APIs
Added 27 mins ago
RL
Lead, Manufacturing Engineer
Rocket Lab
Toronto, ON
- Full-time
At Rocket Lab, we’re not just launching rockets - we're building the future of space. Our Space Systems team builds everything from complete spacecraft and precision payloads to the…
Added 27 mins ago
R
SCADA Transmission Engineer
Raise
Burnaby, BC
- Full-time
The SCADA Transmission Engineer to join our clients team and play a key role in delivering complex, multi-discipline capital projects. In this role, your primary focus will be delivering…
Added 27 mins ago
ED
Sr Software Developer (Lead) - EN 178
Export Development Canada | Exportation et développement Canada
Toronto, ON
- $96,557 to $128,743 CAD / year
- Full-time
Software Engineer 18: Salaries typically range from $96,557 to $128,743 annually, based on qualifications and experiences, plus a performance-based incentive
Added 27 mins ago
ED
Sr Software Developer (Lead) - EN 178
Export Development Canada | Exportation et développement Canada
Calgary, AB
- $96,557 to $128,743 CAD / year
- Full-time
Software Engineer 18: Salaries typically range from $96,557 to $128,743 annually, based on qualifications and experiences, plus a performance-based incentive
Added 27 mins ago
T
Site Reliability Engineer
TechDoQuest
Calgary, AB
- Full-time
The Wealth Management Production Management Site Reliability Engineer position is a highly visible/critical role, which will be a team member of technical SMEs managing the stability and…
Added 27 mins ago
AW
Instrumentation and Control Engineer
Agilus Work Solutions
AB · Edmonton, AB
- Full-time
Lead High-Impact Industrial Projects as a Lead Instrumentation & Controls Engineer
Added 27 mins ago
DC
Ingénieur principal en solutions PLC
Distech Controls
Brossard, QC
- Full-time
Senior PLC Solutions Engineer : Data Center Solutions
Added 27 mins ago
TV
Technical Business Development Director
TandemLaunch Ventures Studio & Funds
Montreal, QC
- Full-time
…hardware-enabled technologies, machine learning, and embedded systems, working directly with researchers from major university labs. We are one of very few venture studios globally with 15+…
Added 27 mins ago
TV
Director of Product Strategy and Commercialization
TandemLaunch Ventures Studio & Funds
Montreal, QC
- Full-time
…hardware-enabled technologies, machine learning, and embedded systems, working directly with researchers from major university labs. We are one of very few venture studios globally with 15+…
Added 27 mins ago
ED
Sr Software Developer (Lead) - EN 178
Export Development Canada | Exportation et développement Canada
Montreal, QC
- $96,557 to $128,743 CAD / year
- Full-time
Software Engineer 18: Salaries typically range from $96,557 to $128,743 annually, based on qualifications and experiences, plus a performance-based incentive
Added 27 mins ago
PG
Senior Software Developer
PEER Group
Kitchener, ON
- $105,000 to $155,000 CAD / year
- Full-time
Want to build high-performance software systems in one of the most complex and specialized industries in the world? We are a founder-led, product-driven software company powering innovation…
Added 27 mins ago
CE
Engineering Specialist Optics
CMC Electronics
Montreal, QC
- Full-time
The technical specialist engineering class 4 works under the supervisor of an engineer or his delegate. Performs a variety of functions associate with the optical, electrical, electronic,…
Added 27 mins ago
IG
Senior Cloud Engineer
Insight Global
Ottawa, ON · Kanata, ON
- $30 to $60 CAD / hour
- Contract
We are seeking a Cloud Engineering Senior Engineer to support the design, development, testing, and deployment of the Telemetry & Observability Platform (TOP). TOP is an enterprise platform…
Added 27 mins ago
TG
Intermediate Draftsperson - Toro
Toro Group of Companies
Concord, ON
- Full-time
Review and implement architectural revisions, ASIs, SKs, and engineer/client comments accurately and in a timely manner.
Added 27 mins ago
FI
Artificial Intelligence Engineer
Flexstaf I.T. Inc.
Canada
- Contract
Integrate AI applications with client systems, APIs, databases, document repositories, and business platforms.

15Rock

Applied AI Engineer - Evaluation & Reliability

Toronto, ON, Canada
Greater Toronto Area, Canada, Canada

Full-time

Apply About employer

Applications open in a new tab.

Job details

Job type: Full-time
Apply by: Apr 7, 2027
Posting language: English
Posted: Oct 9, 2026

Original job description

Applied AI Engineer - Evaluation & ReliabilityThe problem you'd own

15Rock is a financial intelligence firm. Our research goes in front of institutional investors and gets read line by line - by people looking for the one number that doesn't hold up. Every figure has to reconcile across every table and sentence; every claim has to trace to a live source.

We build that research with a multi-agent pipeline on large language models. LLMs write well and count badly: left alone, they produce clean prose that contradicts the table three paragraphs up. Our answer isn't "prompt harder." It's to route every number through deterministic computation and gate every report behind checks that pass or fail - never a model grading its own work. You'd build that layer.

What you'll build

Deterministic evaluation gates that block a report when a defect is present - measured against thresholds set before the run, not a gut feel.
A fact-reconciliation layer that verifies a figure agrees with itself everywhere it appears.
Citation-integrity checks: every reference resolves, every quote is real, every source is live.
Golden-set and regression harnesses that catch quality drift before it ships.
The path from research to production: hardening a validated approach into the pipeline, behind CI, with the traces to prove it works.

What we're looking for

You've shipped LLM systems to production - not demos - where being wrong had a cost.
You've built the evaluations that kept them honest: you think in precision and recall, and you define "good" before you measure it.
Strong Python; you can own backend services and data pipelines yourself.
You reason about the numbers, not just the tokens - enough quantitative or financial literacy to know when a figure is wrong, or the appetite to learn it fast.
Self-directing. You'd rather write a one-page memo than wait for a ticket.

Bonus: model-risk or evaluation-infrastructure work in a regulated setting; LangGraph, DuckDB, CI you owned; a finance or quantitative background.

This role is not

Prompt engineering, a pure research seat, or a place where "the model said so" is an answer.

How we work

A small, senior team with direct access to the founder and research lead. Async by default - we meet on blockers, not to read status aloud. We run on hypotheses and measurable outcomes.

Details

Downtown Toronto, on-site / hybrid. Must be authorized to work in Canada. Early-stage financial intelligence firm, recognized by the Globe and Mail Changemaker Award and selected for Google for Startups.

To Apply: send a short note and something you've built where correctness mattered to Careers@15Rock.com . Tell us about a check you built that caught something a person would have missed.

15Rock is an equal-opportunity employer.

Apply About employer

More about this job

Responsibilities

Build deterministic evaluation gates that block reports with defects, and develop checks for fact consistency, citation validity, and source integrity. Create golden-set and regression testing harnesses, then harden validated approaches for production with CI and traceable evidence.

Requirements

Candidates should have shipped production LLM systems where errors carried real costs and built evaluations using defined quality measures such as precision and recall. Strong Python skills and the ability to own backend services and data pipelines are required, along with quantitative or financial literacy or the ability to learn it quickly.

Skills

Python
LLM Production Systems
Evaluation Design
Precision and Recall
Deterministic Computation
Fact Reconciliation
Citation Integrity
Regression Testing
Backend Services
Data Pipelines
Quantitative Analysis
Financial Literacy
CI
LangGraph
DuckDB

Visa sponsorship

Not detected in the job text

Keywords

Applied AI
Large Language Models
Multi-Agent Pipeline
Python
Deterministic Computation
Evaluation Gates
Precision
Recall
Fact Reconciliation
Citation Integrity
Golden Sets
Regression Testing
Backend Services
Data Pipelines
Production Hardening
Continuous Integration
LangGraph
DuckDB
Model Risk
Evaluation Infrastructure
Financial Intelligence
Quantitative Analysis
Institutional Investors
Source Verification

Report this job