Find work in Canada.

340,765 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 6 mins ago
TC
Adobe Site Reliability Engineer
Tata Consultancy Services
Toronto, ON
- $100,000 to $130,000 CAD / year
- Full-time
TCS is an IT services, consulting, and business solutions organization that has been partnering with many of the world’s largest businesses in their transformation journeys for over 55…
Added 7 mins ago
Field Application Engineer (Integrated Photonics)
Axiomatic_AI
Toronto, ON
We are seeking an experienced Field Application Engineer with expertise across the PIC development lifecycle - from device and circuit design through simulations, tapeout, characterization,…
Added 7 mins ago
I
QA Automation Engineer - ETL/Database
INFOYA
Toronto, ON
- $100,000 to $115,000 CAD / year
- Full-time
experienced Senior Automation Engineer with strong expertise
Added 53 mins ago
MF
SRE/DevOps Engineer
Magnet Forensics
Canada
- Full-time
With employees based around the world, Magnet Forensics has been expanding our global presence. As a part of Magnet Forensics, you can expect to make a difference in the world, no matter…
Added 1 hour ago
1
Applied AI Engineer - Evaluation & Reliability
15Rock
Toronto, ON · Greater Toronto Area, Canada
- Full-time
Applied AI Engineer - Evaluation & ReliabilityThe problem you'd own
Added 3 hours ago
A
Principal Engineer - Open Source Data Platform (ODP)
Acceldata
Kitchener, ON
- Full-time
We are seeking a Principal Engineer with at least 14 years of experience to serve as a technical visionary and thought leader for Acceldata's Open Data Platform (ODP). In this role, you…
Added 3 hours ago
SP
Software Engineering LMTS
Salesforce Partner Canada
Vancouver, BC
- Full-time
The software engineer role at Salesforce encompasses architecture, design, implementation, and testing to ensure we build products right and release them with high quality. Equally…
Added 4 hours ago
K
Senior Software Engineer – DevTest / QA Automation
Kaseya
Toronto, ON
- Full-time
Kaseya is seeking a Senior Software Engineer – DevTest / QA Automation to design, build, and maintain robust automated testing capabilities across enterprise software platforms.
Added 5 hours ago
MA
Senior AI Engineer
Meeru AI Inc
Remote, Canada
- Other
- Remote
…significant pieces of the layer end-to-end, working closely with our Staff AI Engineer and evaluation engineer, and you build per-customer models without leaking the very signal they're…
Added 6 hours ago
ST
AI Engineer Intern
Soft Techniques
Engineer, BC
- Internship
Description Join Soft Techniques as an AI Engineer Intern and work on cutting-edge AI systems used by real businesses. You will collaborate with our engineering team to build and improve AI…
Added 6 hours ago
RT
Senior Data Architect
Ripotek Technologies Inc.
Calgary, AB
- Full-time
…Financial Services, and Public Sector. You'll work directly with C-level stakeholders to translate business requirements into scalable, governed data architectures on Azure, Databricks, and…
Added 6 hours ago
RT
Azure Solutions Consultant
Ripotek Technologies Inc.
Calgary, AB
- Full-time
…Synapse, and Microsoft Fabric. This is a hands-on technical role with client-facing…
Added 6 hours ago
LG
Server Engineer - Unannounced Project
Lightfox Games
Vancouver, BC
- Full-time
Lightfox Games is seeking a talented and enthusiastic Server Engineer to join our Vancouver based team to work on an unannounced mobile game for a world-class IP and Publisher. This role is…
Added 10 hours ago
W
Graduate Technical Consultant, Water Engineer
Worley
Markham, ON
- Full-time
Graduate Technical Consultant I, Water Engineer-in-Training
Added 10 hours ago
W
Technical Consultant II, Water Engineer
Worley
Calgary, AB · Vancouver, BC
- Full-time
The Technical Consultant I, Water Engineer, supports the delivery of high-quality water and wastewater treatment consulting services for industrial clients. Working as part of…
Added 11 hours ago
Staff Software Engineer, Backend Platform
Robinhood
Toronto, ON
…of their careers. We’re a high-performing, fast-moving team with ethics at the center of everything we do. Expectations are high, and so are the…
Added 11 hours ago
M
Senior Software Engineer - Backend
Microsoft
Vancouver, BC
- Full-time
The engineer in this role drives initiatives with company-level impact, ensuring our platform can continue to scale, evolve, and support future Microsoft Ads growth.
Added 12 hours ago
N
Senior Software Engineer, NeMo Helix Core Platform
NVIDIA
Toronto, ON
- Full-time
…high quality architectural RFCs, and to discuss them clearly with the right level of technical detail for the right people (Engineer, Product, Marketing,…
Added 12 hours ago
AM
Senior Firmware Engineer
Advanced Micro Devices, Inc
Markham, ON
- Full-time
We are looking for an experienced and passionate Firmware Engineer to join the NBIO team and drive HW/FW co-development for IP sub-systems. In this role, you will collaborate with hardware,…
Added 12 hours ago
AH
AI/ML Engineer
Axon Health
Ottawa, ON
- Full-time
…safely on their most sensitive data. We're hiring an AI/ML Engineer to build the permission-aware knowledge graph and safety layer behind Axon Health's AI…
Added 13 hours ago
B
Intern, Developer (Winter 2027)
Bombardier
Mississauga, ON
- $17.95 to $31 CAD / hour
- Volunteer
Engineer backend automation scripts and data integration pipelines with Python and SQL, leveraging operational data to drive real-time insights and transformation on the shop floor.
Added 13 hours ago
E
Manager, Developer Experience & AI Enablement
Eyeline
Vancouver, BC
- $205,000 to $305,000 CAD / year
- Full-time
By collaborating with our stakeholders in production, you’ll harness the potential of Eyeline’s “citizen developers” – our community of technically inclined production users – by expanding…
Added 13 hours ago
U
Full-Stack Software Engineer
Ultrassure
Remote, Canada
- Full-time
- Remote
Experience with TypeScript and React or Next.js, alongside Python and backend API development.
Added 14 hours ago
M
Lead AI Solutions Engineer
McKesson
Mississauga, ON
- Full-time
…business consulting, and innovation. The Lead AI Solutions Engineer partners directly with business leaders, operational stakeholders, clinicians, and technical teams to understand…
Added 15 hours ago
GM
Senior Software Developer - Virtualization and SIL Integration
General Motors
Markham, ON
- Full-time
…of Virtualization and Software-in-the-Loop (SIL) Integration Engineer to support our Test Environment Agile Release Train (SIL…

15Rock

Applied AI Engineer - Evaluation & Reliability

Toronto, ON, Canada
Greater Toronto Area, Canada, Canada

Full-time

Apply About employer

Applications open in a new tab.

Job details

Job type: Full-time
Apply by: Apr 7, 2027
Posting language: English
Posted: Oct 9, 2026

Original job description

Applied AI Engineer - Evaluation & ReliabilityThe problem you'd own

15Rock is a financial intelligence firm. Our research goes in front of institutional investors and gets read line by line - by people looking for the one number that doesn't hold up. Every figure has to reconcile across every table and sentence; every claim has to trace to a live source.

We build that research with a multi-agent pipeline on large language models. LLMs write well and count badly: left alone, they produce clean prose that contradicts the table three paragraphs up. Our answer isn't "prompt harder." It's to route every number through deterministic computation and gate every report behind checks that pass or fail - never a model grading its own work. You'd build that layer.

What you'll build

Deterministic evaluation gates that block a report when a defect is present - measured against thresholds set before the run, not a gut feel.
A fact-reconciliation layer that verifies a figure agrees with itself everywhere it appears.
Citation-integrity checks: every reference resolves, every quote is real, every source is live.
Golden-set and regression harnesses that catch quality drift before it ships.
The path from research to production: hardening a validated approach into the pipeline, behind CI, with the traces to prove it works.

What we're looking for

You've shipped LLM systems to production - not demos - where being wrong had a cost.
You've built the evaluations that kept them honest: you think in precision and recall, and you define "good" before you measure it.
Strong Python; you can own backend services and data pipelines yourself.
You reason about the numbers, not just the tokens - enough quantitative or financial literacy to know when a figure is wrong, or the appetite to learn it fast.
Self-directing. You'd rather write a one-page memo than wait for a ticket.

Bonus: model-risk or evaluation-infrastructure work in a regulated setting; LangGraph, DuckDB, CI you owned; a finance or quantitative background.

This role is not

Prompt engineering, a pure research seat, or a place where "the model said so" is an answer.

How we work

A small, senior team with direct access to the founder and research lead. Async by default - we meet on blockers, not to read status aloud. We run on hypotheses and measurable outcomes.

Details

Downtown Toronto, on-site / hybrid. Must be authorized to work in Canada. Early-stage financial intelligence firm, recognized by the Globe and Mail Changemaker Award and selected for Google for Startups.

To Apply: send a short note and something you've built where correctness mattered to Careers@15Rock.com . Tell us about a check you built that caught something a person would have missed.

15Rock is an equal-opportunity employer.

Apply About employer

More about this job

Responsibilities

Build deterministic evaluation gates that block reports with defects, and develop checks for fact consistency, citation validity, and source integrity. Create golden-set and regression testing harnesses, then harden validated approaches for production with CI and traceable evidence.

Requirements

Candidates should have shipped production LLM systems where errors carried real costs and built evaluations using defined quality measures such as precision and recall. Strong Python skills and the ability to own backend services and data pipelines are required, along with quantitative or financial literacy or the ability to learn it quickly.

Skills

Python
LLM Production Systems
Evaluation Design
Precision and Recall
Deterministic Computation
Fact Reconciliation
Citation Integrity
Regression Testing
Backend Services
Data Pipelines
Quantitative Analysis
Financial Literacy
CI
LangGraph
DuckDB

Visa sponsorship

Not detected in the job text

Keywords

Applied AI
Large Language Models
Multi-Agent Pipeline
Python
Deterministic Computation
Evaluation Gates
Precision
Recall
Fact Reconciliation
Citation Integrity
Golden Sets
Regression Testing
Backend Services
Data Pipelines
Production Hardening
Continuous Integration
LangGraph
DuckDB
Model Risk
Evaluation Infrastructure
Financial Intelligence
Quantitative Analysis
Institutional Investors
Source Verification

Report this job