Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 13 mins ago
LM
JR./INTERMEDIATE PROJECT MANAGER AND A SENIOR PROJECT MANAGER
Laframboise Mechanical Electrical Ltd
Long Sault, ON
- Full-time
Added 13 mins ago
LG
JR./INTERMEDIATE PROJECT MANAGER AND A SENIOR PROJECT MANAGER
Laframboise Groupe
Long Sault, ON
- Full-time
Added 13 mins ago
C
Senior QA Automation Engineer – Java & Playwright
Cognizant
Toronto, ON
- $80,000 to $110,000 CAD / year
- Full-time
As a Senior QA Automation Engineer – Java & Playwright, you will make an impact by developing scalable automation solutions and leading end-to-end testing activities for enterprise…
Added 13 mins ago
L
Software Engineer in Test
LTM
Mississauga, ON
- $103,554 to $115,200 CAD / year
- Full-time
Job Role – Software Development Engineer in Test
Added 13 mins ago
WI
Senior Civil Engineer - Ressources
WSP in Canada
Kitimat, BC
- Full-time
To support our continued growth, we are hiring an Senior Civil Engineer. In this role, you’ll help deliver impactful industrial civil engineering projects - from conceptual design to…
Added 13 mins ago
WI
Senior Civil Engineer - Ressources
WSP in Canada
Kitimat, BC
- Full-time
To support our continued growth, we are hiring an Senior Civil Engineer. In this role, you’ll help deliver impactful industrial civil engineering projects - from conceptual design to…
Added 13 mins ago
WI
Senior Professional, Structural Engineering
WSP in Canada
SK · Saskatchewan, Canada
- Full-time
As a Senior Structural Engineer in our Resources Engineering group, you will apply your structural design expertise to heavy industrial and mining projects from conceptual design through…
Added 13 mins ago
WI
Senior Civil Engineer - Ressources
WSP in Canada
Terrace, BC
- Full-time
To support our continued growth, we are hiring an Senior Civil Engineer. In this role, you’ll help deliver impactful industrial civil engineering projects - from conceptual design to…
Added 13 mins ago
WI
Lead Instrumentation & Control Engineer, Resources
WSP in Canada
Sudbury, ON · Greater Sudbury, ON
- Full-time
…Sudbury, Ontario OR Saskatoon, Saskatchewan office as a Lead I&C Engineer, you will support as a senior subject matter expert in the instrumentation and controls engineering for both new…
Added 13 mins ago
WI
Senior Instrumentation & Controls Engineer, Resources
WSP in Canada
Sudbury, ON · Greater Sudbury, ON
- Full-time
…Sudbury, Ontario or Saskatoon Saskatchewan office as a Senior I&C Engineer, you will support as a senior subject matter expert in the instrumentation and controls engineering for both new…
Added 13 mins ago
WI
Lead Instrumentation & Control Engineer, Resources
WSP in Canada
Sudbury, ON · Greater Sudbury, ON
- Full-time
…Sudbury, Ontario OR Saskatoon, Saskatchewan office as a Lead I&C Engineer, you will support as a senior subject matter expert in the instrumentation and controls engineering for both new…
Added 13 mins ago
WI
Senior Civil Engineer - Ressources
WSP in Canada
Sudbury, ON · Greater Sudbury, ON
- Full-time
To support our continued growth, we are hiring an Senior Civil Engineer. In this role, you’ll help deliver impactful industrial civil engineering projects - from conceptual design to…
Added 13 mins ago
WI
Senior Civil Engineer - Ressources
WSP in Canada
Thunder Bay, ON
- Full-time
To support our continued growth, we are hiring an Senior Civil Engineer. In this role, you’ll help deliver impactful industrial civil engineering projects - from conceptual design to…
Added 13 mins ago
WI
Senior Civil Engineer - Ressources
WSP in Canada
Terrace, BC
- Full-time
To support our continued growth, we are hiring an Senior Civil Engineer. In this role, you’ll help deliver impactful industrial civil engineering projects - from conceptual design to…
Added 13 mins ago
WI
Senior Instrumentation & Controls Engineer, Resources
WSP in Canada
Sudbury, ON · Greater Sudbury, ON
- Full-time
…Sudbury, Ontario or Saskatoon Saskatchewan office as a Senior I&C Engineer, you will support as a senior subject matter expert in the instrumentation and controls engineering for both new…
Added 13 mins ago
WI
Senior Civil Engineer - Ressources
WSP in Canada
Sudbury, ON · Greater Sudbury, ON
- Full-time
To support our continued growth, we are hiring an Senior Civil Engineer. In this role, you’ll help deliver impactful industrial civil engineering projects - from conceptual design to…
Added 13 mins ago
A
Senior Software Engineer / Member of Technical Staff (MTS)
Akkodis
Brampton, ON
- Contract
Position: Senior Software Engineer / Member of Technical Staff (MTS)
Added 13 mins ago
T
GenAI Prompt Engineer
TechDoQuest
Toronto, ON
- Contract
-AI Response Validation: Analyze and validate AI-generated outputs to ensure they meet quality standards, are contextually appropriate, and align with user needs.
Added 13 mins ago
WI
Lead Instrumentation & Control Engineer, Resources
WSP in Canada
Saskatoon, SK
- Full-time
…Sudbury, Ontario OR Saskatoon, Saskatchewan office as a Lead I&C Engineer, you will support as a senior subject matter expert in the instrumentation and controls engineering for both new…
Added 13 mins ago
WI
Electrical Engineer in Training (EIT)
WSP in Canada
Markham, ON
- Full-time
…something. This is your chance to make an impact as an Electrical Engineer in Training (EIT) with a team that thrives on curiosity, connection, and…
Added 13 mins ago
WI
Senior Professional, Structural Engineering
WSP in Canada
Saskatoon, SK
- Full-time
As a Senior Structural Engineer in our Resources Engineering group, you will apply your structural design expertise to heavy industrial and mining projects from conceptual design through…
Added 13 mins ago
WI
Electrical Engineer in Training (EIT)
WSP in Canada
Vaughan, ON
- Full-time
…something. This is your chance to make an impact as an Electrical Engineer in Training (EIT) with a team that thrives on curiosity, connection, and…
Added 13 mins ago
WI
Senior Instrumentation & Controls Engineer, Resources
WSP in Canada
Saskatoon, SK
- Full-time
…Sudbury, Ontario or Saskatoon Saskatchewan office as a Senior I&C Engineer, you will support as a senior subject matter expert in the instrumentation and controls engineering for both new…
Added 13 mins ago
WI
Senior Professional, Structural Engineering
WSP in Canada
Regina, SK
- Full-time
As a Senior Structural Engineer in our Resources Engineering group, you will apply your structural design expertise to heavy industrial and mining projects from conceptual design through…
Added 13 mins ago
ES
Client Software Engineer - EA Sports™ FC
EA SPORTS
Vancouver, BC
- Full-time
As a Software Engineer, you will report directly to a Development Director and collaborate closely with producers, designers, artists, and fellow engineers. You will tackle complex…

15Rock

Applied AI Engineer - Evaluation & Reliability

Toronto, ON, Canada
Greater Toronto Area, Canada, Canada

Full-time

Apply About employer

Applications open in a new tab.

Job details

Job type: Full-time
Apply by: Apr 7, 2027
Posting language: English
Posted: Oct 9, 2026

Original job description

Applied AI Engineer - Evaluation & ReliabilityThe problem you'd own

15Rock is a financial intelligence firm. Our research goes in front of institutional investors and gets read line by line - by people looking for the one number that doesn't hold up. Every figure has to reconcile across every table and sentence; every claim has to trace to a live source.

We build that research with a multi-agent pipeline on large language models. LLMs write well and count badly: left alone, they produce clean prose that contradicts the table three paragraphs up. Our answer isn't "prompt harder." It's to route every number through deterministic computation and gate every report behind checks that pass or fail - never a model grading its own work. You'd build that layer.

What you'll build

Deterministic evaluation gates that block a report when a defect is present - measured against thresholds set before the run, not a gut feel.
A fact-reconciliation layer that verifies a figure agrees with itself everywhere it appears.
Citation-integrity checks: every reference resolves, every quote is real, every source is live.
Golden-set and regression harnesses that catch quality drift before it ships.
The path from research to production: hardening a validated approach into the pipeline, behind CI, with the traces to prove it works.

What we're looking for

You've shipped LLM systems to production - not demos - where being wrong had a cost.
You've built the evaluations that kept them honest: you think in precision and recall, and you define "good" before you measure it.
Strong Python; you can own backend services and data pipelines yourself.
You reason about the numbers, not just the tokens - enough quantitative or financial literacy to know when a figure is wrong, or the appetite to learn it fast.
Self-directing. You'd rather write a one-page memo than wait for a ticket.

Bonus: model-risk or evaluation-infrastructure work in a regulated setting; LangGraph, DuckDB, CI you owned; a finance or quantitative background.

This role is not

Prompt engineering, a pure research seat, or a place where "the model said so" is an answer.

How we work

A small, senior team with direct access to the founder and research lead. Async by default - we meet on blockers, not to read status aloud. We run on hypotheses and measurable outcomes.

Details

Downtown Toronto, on-site / hybrid. Must be authorized to work in Canada. Early-stage financial intelligence firm, recognized by the Globe and Mail Changemaker Award and selected for Google for Startups.

To Apply: send a short note and something you've built where correctness mattered to Careers@15Rock.com . Tell us about a check you built that caught something a person would have missed.

15Rock is an equal-opportunity employer.

Apply About employer

More about this job

Responsibilities

Build deterministic evaluation gates that block reports with defects, and develop checks for fact consistency, citation validity, and source integrity. Create golden-set and regression testing harnesses, then harden validated approaches for production with CI and traceable evidence.

Requirements

Candidates should have shipped production LLM systems where errors carried real costs and built evaluations using defined quality measures such as precision and recall. Strong Python skills and the ability to own backend services and data pipelines are required, along with quantitative or financial literacy or the ability to learn it quickly.

Skills

Python
LLM Production Systems
Evaluation Design
Precision and Recall
Deterministic Computation
Fact Reconciliation
Citation Integrity
Regression Testing
Backend Services
Data Pipelines
Quantitative Analysis
Financial Literacy
CI
LangGraph
DuckDB

Visa sponsorship

Not detected in the job text

Keywords

Applied AI
Large Language Models
Multi-Agent Pipeline
Python
Deterministic Computation
Evaluation Gates
Precision
Recall
Fact Reconciliation
Citation Integrity
Golden Sets
Regression Testing
Backend Services
Data Pipelines
Production Hardening
Continuous Integration
LangGraph
DuckDB
Model Risk
Evaluation Infrastructure
Financial Intelligence
Quantitative Analysis
Institutional Investors
Source Verification

Report this job