Find work in Canada.

344,748 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 26 mins ago
A
Staff Software Engineer - Open Source Data Platform (ODP)
Acceldata
Kitchener, ON
- Full-time
We are seeking a Staff Engineer with at least 9 years of experience in distributed systems or data platform development. In this role, you will be a technical leader responsible for…
Added 1 hour ago
RC
Senior AI Engineer
Realtek Consulting LLC
Toronto, ON
- Contract
Job Location: Toronto,Ontario-3days per week in office Required-Hybrid
Added 1 hour ago
B
Senior Manager, Data Engineer, Applied AI Data Ingestion & Integration
BMO
Toronto, ON · FCP
- Full-time
HYBRID ROLE. This is an Individual Contributor role for a professional who can run initiatives and drive the results.
Added 2 hours ago
KG
Senior Platform Engineer
Kinross Gold Corporation
Toronto, ON
- Full-time
…are seeking a highly technical and hands-on Senior Platform Engineer to drive the evolution of our hybrid infrastructure ecosystem spanning on-premises datacenters, cloud platforms,…
Added 2 hours ago
Agentic AI Architect, Kitchener, ON ( Hybrid)
Siemens
Kitchener, ON
…ever-changing challenges, customer needs, and questions from colleagues with clever concepts and creativity. We embrace change and work with curious minds re-inventing the future of work.…
Added 2 hours ago
Associate Technical Services Engineer II
MongoDB
Toronto, ON
This role is based in Toronto and follows a hybrid working model, with a minimum expectation of three days per week in the office.
Added 2 hours ago
IG
Machine Learning Engineer
IMCS Group
ON · Ontario, Canada
- Contract
We are looking for a highly experienced Senior AWS SageMaker / MLOps Engineer with 6+ years of overall IT experience and strong expertise in building, deploying, and operationalizing…
Added 2 hours ago
EC
Ingénieur(e) en réseaux de puissance / Power Systems Engineer
Eaton Corporation
Brossard, QC
- Full-time
Eaton’s CYME International T&D is an industry-leading provider of grid planning and simulation software solutions with an established reputation for customer responsiveness and technical…
Added 3 hours ago
SL
Principal Software Engineer, MarTech Platforms & Digital Enablement
Sun Life
Toronto, ON · Waterloo, ON · Sun Life Toronto Atria
- Full-time
Strong hands-on experience with: Java / Spring Boot / Python JavaScript / TypeScript APIs and Microservices Docker and Kubernetes
Added 3 hours ago
E
Engineer in Training I
Enbridge
Edmonton, AB
- Full-time
…excellence and innovation. We are seeking a Pipeline Integrity Engineer-in-Training I to join our…
Added 3 hours ago
Specialist Systems, IT
City of Mississauga
Mississauga, ON
This role requires hands-on experience with AWS services including ECS, EC2, RDS, S3, CloudFront, Lambda, Route 53, CloudWatch, IAM and networking services, as well as Windows and Linux…
Added 3 hours ago
FD
AI Engineer
Frontier Dental CA
Markham, ON
- Full-time
We’re seeking an AI Engineer to help shape the future of Frontier Dental through practical AI adoption and business automation.
Added 3 hours ago
CO
Specialist Systems, IT
City of Mississauga
Mississauga, ON
- Full-time
This role requires hands-on experience with AWS services including ECS, EC2, RDS, S3, CloudFront, Lambda, Route 53, CloudWatch, IAM and networking services, as well as Windows and Linux…
Added 3 hours ago
T
Machine Learning Engineer
TEEMA
Toronto, ON
- $115,000 to $120,000 CAD / year
- Full-time
Location: Toronto, ON – Hybrid, minimum 2 days onsite per week, with the possibility of 3 days
Added 3 hours ago
W
AI Engineer/Developer
W3Global
Toronto, ON
- $60,000 to $68,000 CAD / year
- Full-time
Develop AI-powered applications using Python, LangChain, vector databases, and cloud services.
Added 3 hours ago
SL
Principal Software Engineer, MarTech Platforms & Digital Enablement
Sun Life
Toronto, ON · Waterloo, ON · Sun Life Toronto Atria
- Full-time
Strong hands-on experience with: Java / Spring Boot / Python JavaScript / TypeScript APIs and Microservices Docker and Kubernetes
Added 4 hours ago
SR
Senior Software Engineer
Sage Recruiting Inc.
Toronto, ON
- $180,000 to $220,000 CAD / year
- Full-time
They’re hiring multiple engineers, all full-stack with a backend focus, to work on the core product. You could also mix in some ETL work or work on their RAG engine in Python if that’s of…
Added 4 hours ago
VC
Staff Software Engineer
Valsoft Corporation
Remote, Canada
- Full-time
- Remote
We're hiring a Staff Software Engineer to own architecture and design decisions for our platform, including the modernization of legacy enterprise systems into modern, cloud-capable…
Added 4 hours ago
WT
Data Engineer
Wissen Technology
Montreal, QC
- Full-time
We are looking for a Data Engineer with strong backend engineering fundamentals and hands‑on experience building scalable, high‑performance data pipelines. The ideal candidate comes from a…
Added 4 hours ago
V
Python Software Engineer
Varonis
Vancouver, BC · Greater Vancouver, BC
- Full-time
Senior Software Engineer – Secure AI Platform (Vancouver, On-Site)
Added 5 hours ago
Product Security Engineer
Asana
Vancouver, BC
We're looking for a Product Security Engineer to join our Product Security team in Vancouver. You'll be a key member of our security engineering team, partnering directly with product and…
Added 5 hours ago
F
Technical Consultant (Data & AI)
Fastloop
Vancouver, BC
- Full-time
Backend: Advanced Python (FastAPI/Django). Familiarity with Document AI or unstructured data parsing is a plus.
Added 5 hours ago
W
Technical Consultant I, Water Engineer
Worley
Vancouver, BC
- Full-time
The Technical Consultant I, Water Engineer, supports the delivery of high-quality water and wastewater treatment consulting services for industrial clients. Working as part of…
Added 5 hours ago
W
Technical Consultant I, Water Engineer
Worley
Calgary, AB
- Full-time
The Technical Consultant I, Water Engineer, supports the delivery of high-quality water and wastewater treatment consulting services for industrial clients. Working as part of…
Added 5 hours ago
S
Analytics Engineer - Consultant or Senior Consultant
Slalom
Toronto, ON · Greater Toronto Area, Canada
- Full-time
Proficiency in SQL and/or Python; experience with semantic layer tools and analytics platforms (e.g., dbt, Looker, Power BI, Tableau, Sigma)

15Rock

Applied AI Engineer - Evaluation & Reliability

Toronto, ON, Canada
Greater Toronto Area, Canada, Canada

Full-time

Apply About employer

Applications open in a new tab.

Job details

Job type: Full-time
Apply by: Apr 7, 2027
Posting language: English
Posted: Oct 9, 2026

Original job description

Applied AI Engineer - Evaluation & ReliabilityThe problem you'd own

15Rock is a financial intelligence firm. Our research goes in front of institutional investors and gets read line by line - by people looking for the one number that doesn't hold up. Every figure has to reconcile across every table and sentence; every claim has to trace to a live source.

We build that research with a multi-agent pipeline on large language models. LLMs write well and count badly: left alone, they produce clean prose that contradicts the table three paragraphs up. Our answer isn't "prompt harder." It's to route every number through deterministic computation and gate every report behind checks that pass or fail - never a model grading its own work. You'd build that layer.

What you'll build

Deterministic evaluation gates that block a report when a defect is present - measured against thresholds set before the run, not a gut feel.
A fact-reconciliation layer that verifies a figure agrees with itself everywhere it appears.
Citation-integrity checks: every reference resolves, every quote is real, every source is live.
Golden-set and regression harnesses that catch quality drift before it ships.
The path from research to production: hardening a validated approach into the pipeline, behind CI, with the traces to prove it works.

What we're looking for

You've shipped LLM systems to production - not demos - where being wrong had a cost.
You've built the evaluations that kept them honest: you think in precision and recall, and you define "good" before you measure it.
Strong Python; you can own backend services and data pipelines yourself.
You reason about the numbers, not just the tokens - enough quantitative or financial literacy to know when a figure is wrong, or the appetite to learn it fast.
Self-directing. You'd rather write a one-page memo than wait for a ticket.

Bonus: model-risk or evaluation-infrastructure work in a regulated setting; LangGraph, DuckDB, CI you owned; a finance or quantitative background.

This role is not

Prompt engineering, a pure research seat, or a place where "the model said so" is an answer.

How we work

A small, senior team with direct access to the founder and research lead. Async by default - we meet on blockers, not to read status aloud. We run on hypotheses and measurable outcomes.

Details

Downtown Toronto, on-site / hybrid. Must be authorized to work in Canada. Early-stage financial intelligence firm, recognized by the Globe and Mail Changemaker Award and selected for Google for Startups.

To Apply: send a short note and something you've built where correctness mattered to Careers@15Rock.com . Tell us about a check you built that caught something a person would have missed.

15Rock is an equal-opportunity employer.

Apply About employer

More about this job

Responsibilities

Build deterministic evaluation gates that block reports with defects, and develop checks for fact consistency, citation validity, and source integrity. Create golden-set and regression testing harnesses, then harden validated approaches for production with CI and traceable evidence.

Requirements

Candidates should have shipped production LLM systems where errors carried real costs and built evaluations using defined quality measures such as precision and recall. Strong Python skills and the ability to own backend services and data pipelines are required, along with quantitative or financial literacy or the ability to learn it quickly.

Skills

Python
LLM Production Systems
Evaluation Design
Precision and Recall
Deterministic Computation
Fact Reconciliation
Citation Integrity
Regression Testing
Backend Services
Data Pipelines
Quantitative Analysis
Financial Literacy
CI
LangGraph
DuckDB

Visa sponsorship

Not detected in the job text

Keywords

Applied AI
Large Language Models
Multi-Agent Pipeline
Python
Deterministic Computation
Evaluation Gates
Precision
Recall
Fact Reconciliation
Citation Integrity
Golden Sets
Regression Testing
Backend Services
Data Pipelines
Production Hardening
Continuous Integration
LangGraph
DuckDB
Model Risk
Evaluation Infrastructure
Financial Intelligence
Quantitative Analysis
Institutional Investors
Source Verification

Report this job