Find work in Canada.

327,774 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Save this search

Added 2 days ago
Senior Forward-Deployed AI Engineer
Bell
Toronto, ON
As a Senior Forward-Deployed AI Engineer, you will own engagements end-to-end: scoping the problem, choosing the right approach across automation, classical ML, and GenAI/agents, and taking…
Added 17 days ago
Middle/Senior Forward Deployed Engineer (Python, AI)
Exadel
Canada
For the Senior Forward Deployed Engineer level, senior-level experience, including technical delivery leadership across multidisciplinary teams or pods.
Added 13 days ago
Senior Forward Deployed Engineer - Okta for AI Agents
Okta
Toronto, ON
Engage senior leadership. Brief the CISO, CIO, identity leaders, Chief AI Officer, and principal architects. Translate token-exchange flows into board-level agent risk, and AI governance…
Added 6 hours ago
Principal Forward Deployed Engineer
''
Toronto, ON
Full-time · Other
Position Title: Principal Forward Deployed Engineer
Added 5 days ago
Senior Software Engineer (Conductor)
PocketHealth
Toronto, ON
Full-time
As a Senior Software Engineer, you are a full-stack software engineer who works directly alongside healthcare providers to get Conductor live and to make it matter. You take a signed…
Added 1 day ago
Head of Forward Deployed Engineering (FDE)
Beacon Holdings
Toronto, ON
Full-time
Built and led a forward-deployed, solutions, or embedded engineering function at a company where the product was an AI or data platform; you have run the transition from "every engineer…
Added 2 days ago
Head of Forward Deployed Engineering (FDE)
Beacon Software
Toronto, ON
Full-time
Built and led a forward-deployed, solutions, or embedded engineering function at a company where the product was an AI or data platform; you have run the transition from "every engineer…
Added 3 days ago
Forward Deployed Engineer (FDE) (Mid/Senior Level)
Salesforce Partner Canada
Toronto, ON
Full-time
Collaborative Delivery: You'll work alongside Senior Forward Deployed Engineers and a Deployment Strategist. You'll own real components, benefit from structured mentorship from people…
Added 3 months ago
Merchant Success Manager
FUEGO.io
NB · York, NB
Full-time
You do not write code. You speak Shopify fluently enough that customers trust you, and you know exactly when to pull in our Forward Deployed Design Engineering team to do the technical work.
Added 5 days ago
Senior Forward Deployed Engineer (Hybrid, Canada)
Unit8
Montreal, QC
Full-time
…Unit8 is expanding into the Canadian market. We are seeking a Senior Forward Deployed Engineer who will work directly with customers, owning delivery strategy and…
Added 6 days ago
Senior Software Engineer, Full Stack, Applied AI Agent Studio
Google
Waterloo, ON
Full-time
…mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as…
Added 6 days ago
Senior Software Engineer, Full Stack, Applied AI Agent Studio
Google
Toronto, ON
Full-time
…mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as…
Added 11 days ago
Expert Forward Deployment Engineer
Luxoft
Mississauga, ON
Full-time
Forward Deployed Engineers will conduct targeted, hands-on engagements with development teams that are facing challenges with vulnerability remediation, automated testing, legacy…
Added 11 days ago
Senior Forward Deployment Engineer
Luxoft
Mississauga, ON
Full-time
Forward Deployed Engineers will conduct targeted, hands-on engagements with development teams that are facing challenges with vulnerability remediation, automated testing, legacy…
Added 12 days ago
Senior OT Network and Security Engineer
CBRE
Toronto, ON
Full-time
As a CBRE Senior Network and Security Engineer, you’ll be responsible for the design, delivery, and operational excellence of building-level IT and OT networks across the portfolio.
Added 16 days ago
Staff Technical Architect
Appnovation
Toronto, ON
Full-time
As a Senior Technical Architect (Director level), you will join a highly motivated and experienced team as the technical leader accountable for crafting the technical strategy and…
Added 17 days ago
Senior Software Engineer, Conductor
PocketHealth
Toronto, ON
Full-time
As a Senior Software Engineer, you are a full-stack software engineer who works directly alongside healthcare providers to get Conductor live and to make it matter. You take a signed…
Added 16 days ago
Staff Technical Architect
Appnovation Technologies
Toronto, ON
As a Senior Technical Architect (Director level), you will join a highly motivated and experienced team as the technical leader accountable for crafting the technical strategy and…
Added 18 days ago
Agentic AI Engineer/Architect - FTA
SIA Innovations INC
Montreal, QC
FORWARD - DEPLOYED Engineer · Montréal, QC · Full-Time, Permanent · 100% On-Site · Start Date: September 14, 2026
Added 19 days ago
Senior Forward Deployed Engineer (Gemini Enterprise)
Artefact
Montreal, QC
Artefact is looking for a Senior Deployed AI Engineer specialized in Gemini Enterprise and the Google AI stack: an engineer who works embedded with our clients and takes AI products from…
Added 1 month ago
Software Engineer, Applied AI
St. Joseph Communications
Toronto, ON
$110,000 to $130,000 CAD / year
SJC Media is building the AI layer for its Digital Operations. As the founding Software Engineer, Applied AI, you own that layer end-to-end: from agentic workflows, CRM integrations,…
Added 27 days ago
Senior Agentic Engineer
TribalScale
Toronto, ON
Full-time
TribalScale is looking for a Forward Deployed Engineer to work at the intersection of software engineering, artificial intelligence, product development, and client delivery.
Added 24 days ago
Forward Deployed Engineer (FDE) (Mid/Senior Level)
Salesforce
Toronto, ON · Canada - Toronto
Full-time
Collaborative Delivery: You'll work alongside Senior Forward Deployed Engineers and a Deployment Strategist. You'll own real components, benefit from structured mentorship from people…
Added 24 days ago
QA Engineer (AI Systems)
Nexxa.AI
Remote, Canada
Full-time · Remote
We're looking for a Lead / Senior / Staff QA Engineer to own quality for Nexxa's AI agent systems - products that plan, call tools, and take multi-step actions autonomously in industrial…
Added 1 month ago
Senior Forward Deployment Engineer
Luxoft
Mississauga, ON
Full-time
Forward Deployed Engineers will conduct targeted, hands-on engagements with development teams that are facing challenges with vulnerability remediation, automated testing, legacy…

Nexxa.AI

QA Engineer (AI Systems)

Remote, Canada

Job type: Full-time
Workplace: Remote
Posting language: English
Posted: Aug 27, 2026

Apply About employer

Applications open in a new tab.

Original job description

Nexxa is building the best AI systems for heavy industries - enabling machines, systems and operations to think, decide and act autonomously across manufacturing, large-scale infrastructure, logistics and legacy environments.

Our mission is to translate deep technical breakthroughs into operational reality, solving some of the hardest systems-level problems in industry.

Role Overview

We're looking for a Lead / Senior / Staff QA Engineer to own quality for Nexxa's AI agent systems - products that plan, call tools, and take multi-step actions autonomously in industrial environments. This isn't traditional UI testing: you'll be designing evaluation frameworks for non-deterministic, tool-using systems, building golden datasets, catching regressions in reasoning quality, and stress-testing agent behavior under adversarial and real-world edge-case conditions.

You'll work closely with ML engineers, backend engineers, and Forward Deployed Engineers to define what "good" looks like for an agent operating in high-stakes industrial settings, then build the infrastructure and processes to measure it continuously.

Key Responsibilities

Design and build evaluation harnesses and regression suites for LLM-based agents, covering reasoning quality, tool-call correctness, task completion, and multi-turn coherence.
Develop golden datasets and labeled test sets, including edge cases, ambiguous inputs, and adversarial prompts specific to industrial and operational contexts.
Define and track quality metrics beyond simple accuracy - groundedness, hallucination rate, task success rate, latency/cost tradeoffs, and safety violations.
Build automated pipelines that run evals on every model, prompt, or tool-integration change, and integrate them into CI/CD.
Conduct structured red-teaming and adversarial testing (prompt injection, jailbreaks, tool misuse, unsafe actions) in partnership with security teams.
Test agent behavior across the full action loop - planning, tool selection, tool execution, error recovery, and final output - not just the final response.
Investigate and triage failures where the root cause could be the model, the prompt, the tool/API, or the orchestration logic.
Partner with ML and backend engineers to translate eval failures into actionable, reproducible bug reports.
Establish quality bars and sign-off criteria for new agent capabilities before they reach customer environments.
Mentor other engineers on testing strategies specific to probabilistic, LLM-driven systems.
Advocate for testability and observability in agent architecture from day one.

Qualifications

5+ years in QA/SDET roles, with demonstrated ownership of test strategy for complex systems.
Hands-on experience testing LLM-based products, chatbots, or AI agents - you understand why traditional deterministic test assertions break down for generative systems.
Practical experience with eval frameworks or tooling (e.g., promptfoo, DeepEval, RAGAS, LangSmith) or a track record of building your own.
Strong scripting/programming ability (Python preferred) to build test automation, data pipelines, and eval tooling.
Understanding of how LLM agents work: prompting, tool/function calling, context management, RAG, memory, and orchestration frameworks.
Experience designing test data and labeled datasets, including sourcing, sampling, and managing dataset drift over time.
Familiarity with LLM-specific failure modes: hallucination, prompt injection, context poisoning, tool misuse, goal drift, and non-determinism.
Comfortable operating in ambiguity - defining what "correct" means for a task when there's no single right answer.
Strong written communication skills for turning fuzzy quality signals into clear, actionable findings for engineering and product stakeholders.

Preferred

Experience with human-in-the-loop evaluation workflows (labeling pipelines, inter-rater reliability, rubric design).
Background in ML/data science sufficient to read model evals and statistical significance.
Experience red-teaming or doing adversarial/security testing on ML systems.
Familiarity with observability/tracing tools for LLM applications (e.g., LangSmith, Arize, Langfuse, Weights & Biases).
Experience testing AI systems in industrial, IoT, or operational technology (OT) environments.
Prior experience setting up eval infrastructure from scratch at a startup or fast-moving team.
What We're Looking For A QA engineer who wants to define what quality means for autonomous, real-world AI systems.
Someone who can build rigorous evaluation infrastructure for problems that don't have a single right answer.
A systems thinker who enjoys turning ambiguous agent behavior into measurable, trustworthy signals.
A strong collaborator who partners well with ML engineers, backend engineers, and Forward Deployed teams.

Why Join Nexxa.AI?

Innovative Environment: Play a critical role in transforming heavy industries through groundbreaking AI and automation technologies.

Collaborative Culture: Be part of a team that values innovation, discipline, and continuous improvement.

Professional Growth: Benefit from significant opportunities for career development and advancement.

Competitive Compensation: Enjoy a comprehensive salary and equity package reflective of your expertise and contributions.

If you're passionate about AI quality and eager to help define what trustworthy autonomous systems look like in heavy industry, we'd love to connect.

Apply About employer

More about this job

Responsibilities

Design and build evaluation harnesses and regression suites for LLM-based agents to ensure reasoning quality and tool-call correctness. Develop automated pipelines and conduct structured red-teaming to stress-test agent behavior in high-stakes industrial environments.

Requirements

Requires 5+ years of experience in QA/SDET roles with hands-on experience testing LLM-based products or AI agents. Strong programming skills in Python and familiarity with evaluation frameworks like promptfoo, DeepEval, or LangSmith are essential.

Skills

QA Engineering
LLM
Python
Test automation
Evaluation frameworks
Prompt engineering
Red-teaming
Adversarial testing
Data pipelines
CI/CD
Tool calling
RAG
Observability
System testing
Debugging
Mentorship

Benefits

Competitive compensation
Equity package
Professional growth opportunities

Visa sponsorship

Not detected in the job text

Keywords

QA Engineer
AI Systems
LLM
Python
Promptfoo
DeepEval
RAGAS
LangSmith
CI/CD
Red-teaming
Adversarial testing
Hallucination
Tool calling
Orchestration
Data pipelines
Industrial automation
IoT
Observability
Tracing
Arize
Langfuse
Weights & Biases
Regression testing
Test strategy
Software engineering
Machine learning
System architecture
Quality assurance
Edge cases
Autonomous systems

Report this job