Find work in Canada.

343,920 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 9 mins ago
LC
Programming Engineer
Linamar Corporation
Guelph, ON
- Full-time
The Programming Engineer is responsible for designing, developing, and supporting software and system solutions that enhance manufacturing and engineering operations. This role owns key…
Added 9 mins ago
LC
Mechanical Design Engineer
Linamar Corporation
Guelph, ON
- Full-time
Develop and design new products, components, and systems, ensuring they meet performance, safety, and regulatory requirements.
Added 24 mins ago
M
Electronic Circuit Automation and DevOps Developer/Engineer
MDA
Sainte-Anne-de-Bellevue, QC
- Full-time
As an Electronic Circuit Design Automation and DevOps Engineer, you will provide support to electronics development teams for the design automation of various activities related to the…
Added 24 mins ago
M
Electronic Circuit Automation and DevOps Developer/Engineer
MDA
Sainte-Anne-de-Bellevue, QC
- Full-time
As an Electronic Circuit Design Automation and DevOps Professional/Engineer, you will provide support to electronics development teams for the design automation of various activities…
Added 24 mins ago
P
Senior Software Engineer
Pebl
Remote, Canada
- $122,835 to $180,000 CAD / year
- Full-time
- Remote
Build observability and evaluation into our AI systems so we can see how agents behave in production and improve them with evidence.
Added 37 mins ago
RC
Senior AI Engineer
Realtek Consulting LLC
Toronto, ON
- Contract
6-10 years building production software, including recent hands-on delivery of AI or large-language-model systems beyond prototypes.
Added 37 mins ago
RC
Manufacturing Engineer
Randstad Canada
Calgary, AB · Greater Calgary Metropolitan Area
- Full-time
…a hands-on, highly skilled Lead Manufacturing & Sustaining Engineer to bridge the gap between product development, operations, and quality. In this technical leadership role, you will play…
Added 37 mins ago
GU
AI Agentic Systems & Automation Engineer
GenCore Utilities Ltd.
Markham, ON
- $100,000 to $140,000 CAD / year
- Full-time
GenCore Utilities Ltd. is seeking an AI Agentic Systems & Automation Engineer to design, build, and deploy AI solutions that improve engineering, operations, project delivery, and business…
Added 55 mins ago
SL
Senior Software Engineer, Operations
Sun Life
Toronto, ON · Waterloo, ON · Sun Life Waterloo King
- Full-time
We are looking for a highly motivated Senior Software Engineer with at least 10 years of Java application support, analysis, and development experience. The successful candidate will have…
Added 55 mins ago
B
Senior Manager, Data Engineer, Applied AI Data Ingestion & Integration
BMO
Toronto, ON · FCP
- Full-time
…partnerships with third-party providers for validating and adopting production-grade solutions. Your work will directly support the development of AI-ready datasets, multimodal document…
Added 1 hour ago
S
Head of Data Systems (Google Stack)
Stacktics
Toronto, ON
- Full-time
The Head of Data Systems (Google Stack) will play a key role at Stacktics, required to help create and maintain industry-leading quality and efficiency of service and software delivery.
Added 1 hour ago
Agentic AI Architect, Kitchener, ON ( Hybrid)
Siemens
Kitchener, ON
…technical design of our Agentic AI platform and ensure the agent systems we build are coherent, scalable, and production-ready. The role focuses on architecture and system design -…
Added 1 hour ago
ED
Sr Software Developer (Lead) - EN 178
Export Development Canada | Exportation et développement Canada
Remote, Canada
- Full-time
- Remote
Software Engineer 18: Salaries typically range from $96,557 to $128,743 annually, based on qualifications and experiences, plus a performance-based incentive.
Added 1 hour ago
ED
Sr Software Developer (Lead) - EN 178
Export Development Canada | Exportation et développement Canada
Remote, Canada
- Full-time
- Remote
Software Engineer 18: Salaries typically range from $96,557 to $128,743 annually, based on qualifications and experiences, plus a performance-based incentive.
Added 1 hour ago
ED
Sr Software Developer (Lead) - EN 178
Export Development Canada | Exportation et développement Canada
Remote, Canada
- Full-time
- Remote
Software Engineer 18: Salaries typically range from $96,557 to $128,743 annually, based on qualifications and experiences, plus a performance-based incentive.
Added 1 hour ago
ED
Sr Software Developer (Lead) - EN 178
Export Development Canada | Exportation et développement Canada
Remote, Canada
- Full-time
- Remote
Software Engineer 18: Salaries typically range from $96,557 to $128,743 annually, based on qualifications and experiences, plus a performance-based incentive.
Added 1 hour ago
ED
Sr Software Developer (Lead) - EN 178
Export Development Canada | Exportation et développement Canada
Remote, Canada
- Full-time
- Remote
Software Engineer 18: Salaries typically range from $96,557 to $128,743 annually, based on qualifications and experiences, plus a performance-based incentive.
Added 1 hour ago
CL
Senior Cloud DevOps Engineer
Charger Logistics Inc
Brampton, ON
- Full-time
The Senior Cloud DevOps Engineer will design and implement enterprise-scale cloud infrastructure, drive automation strategies, and mentor engineering teams while architecting…
Added 1 hour ago
Associate Technical Services Engineer II
MongoDB
Toronto, ON
If you have an operations background, we prefer experience administering large-scale production environments, including hardware, operating systems (e.g. Linux, Windows), networks…
Added 1 hour ago
S
Software Engineer, Streaming Platform
Sentry
Remote, Canada
- $163,000 to $200,000 CAD / year
- Full-time
- Remote
As part of this team, you will work on challenges at the intersection of distributed systems, real-time data processing, and developer experience. You will help us create a self-service…
Added 1 hour ago
AU
Maintenance Technician (Fifth Class Power Engineer)
Alsco Uniforms
Surrey, BC
- Full-time
…and cutting, hydraulics, pneumatics, mechanical, and other systems. This position reports to the Chief Engineer. Start rate…
Added 1 hour ago
CL
Senior Cloud DevOps Engineer
Charger Logistics Inc.
Brampton, ON
- Full-time
The Senior Cloud DevOps Engineer will design and implement enterprise-scale cloud infrastructure, drive automation strategies, and mentor engineering teams while architecting…
Added 1 hour ago
Logistics Engineering Specialist
Siemens
Concord, ON
The Logistics Engineer is responsible for analyzing, designing, implementing, and continuously improving material-flow and logistics processes across warehouse, receiving, production,…
Added 2 hours ago
ED
Sr Software Developer (Lead) - EN 178
Export Development Canada | Exportation et développement Canada
Ottawa, ON
- $96,557 to $128,743 CAD / year
- Full-time
Software Engineer 18: Salaries typically range from $96,557 to $128,743 annually, based on qualifications and experiences, plus a performance-based incentive
Added 2 hours ago
Manufacturing Technician - Test Operator
INNIO
Welland, ON
We are seeking a skilled and motivated Test Technician to join our Production Testing team in Welland Ontario Canada, reporting to the Lead Production Test Engineer. In this role, you will…

15Rock

Applied AI Engineer - Evaluation & Reliability

Toronto, ON, Canada
Greater Toronto Area, Canada, Canada

Full-time

Apply About employer

Applications open in a new tab.

Job details

Job type: Full-time
Apply by: Apr 7, 2027
Posting language: English
Posted: Oct 9, 2026

Original job description

Applied AI Engineer - Evaluation & ReliabilityThe problem you'd own

15Rock is a financial intelligence firm. Our research goes in front of institutional investors and gets read line by line - by people looking for the one number that doesn't hold up. Every figure has to reconcile across every table and sentence; every claim has to trace to a live source.

We build that research with a multi-agent pipeline on large language models. LLMs write well and count badly: left alone, they produce clean prose that contradicts the table three paragraphs up. Our answer isn't "prompt harder." It's to route every number through deterministic computation and gate every report behind checks that pass or fail - never a model grading its own work. You'd build that layer.

What you'll build

Deterministic evaluation gates that block a report when a defect is present - measured against thresholds set before the run, not a gut feel.
A fact-reconciliation layer that verifies a figure agrees with itself everywhere it appears.
Citation-integrity checks: every reference resolves, every quote is real, every source is live.
Golden-set and regression harnesses that catch quality drift before it ships.
The path from research to production: hardening a validated approach into the pipeline, behind CI, with the traces to prove it works.

What we're looking for

You've shipped LLM systems to production - not demos - where being wrong had a cost.
You've built the evaluations that kept them honest: you think in precision and recall, and you define "good" before you measure it.
Strong Python; you can own backend services and data pipelines yourself.
You reason about the numbers, not just the tokens - enough quantitative or financial literacy to know when a figure is wrong, or the appetite to learn it fast.
Self-directing. You'd rather write a one-page memo than wait for a ticket.

Bonus: model-risk or evaluation-infrastructure work in a regulated setting; LangGraph, DuckDB, CI you owned; a finance or quantitative background.

This role is not

Prompt engineering, a pure research seat, or a place where "the model said so" is an answer.

How we work

A small, senior team with direct access to the founder and research lead. Async by default - we meet on blockers, not to read status aloud. We run on hypotheses and measurable outcomes.

Details

Downtown Toronto, on-site / hybrid. Must be authorized to work in Canada. Early-stage financial intelligence firm, recognized by the Globe and Mail Changemaker Award and selected for Google for Startups.

To Apply: send a short note and something you've built where correctness mattered to Careers@15Rock.com . Tell us about a check you built that caught something a person would have missed.

15Rock is an equal-opportunity employer.

Apply About employer

More about this job

Responsibilities

Build deterministic evaluation gates that block reports with defects, and develop checks for fact consistency, citation validity, and source integrity. Create golden-set and regression testing harnesses, then harden validated approaches for production with CI and traceable evidence.

Requirements

Candidates should have shipped production LLM systems where errors carried real costs and built evaluations using defined quality measures such as precision and recall. Strong Python skills and the ability to own backend services and data pipelines are required, along with quantitative or financial literacy or the ability to learn it quickly.

Skills

Python
LLM Production Systems
Evaluation Design
Precision and Recall
Deterministic Computation
Fact Reconciliation
Citation Integrity
Regression Testing
Backend Services
Data Pipelines
Quantitative Analysis
Financial Literacy
CI
LangGraph
DuckDB

Visa sponsorship

Not detected in the job text

Keywords

Applied AI
Large Language Models
Multi-Agent Pipeline
Python
Deterministic Computation
Evaluation Gates
Precision
Recall
Fact Reconciliation
Citation Integrity
Golden Sets
Regression Testing
Backend Services
Data Pipelines
Production Hardening
Continuous Integration
LangGraph
DuckDB
Model Risk
Evaluation Infrastructure
Financial Intelligence
Quantitative Analysis
Institutional Investors
Source Verification

Report this job