Jobs in Canada

Email me new jobs Save this search

Added 16 days ago
Développeur Senior SRE
Astek
Montreal, QC
Full-time
Senior Developer SRE "Site Reliability Engineer"
Added 16 days ago
Senior Software Engineer -Waterloo
PointClickCare
Waterloo, ON
$142,000 to $158,000 CAD / year
Full-time
Experience writing clean code that performs well at scale using Java
Added 16 days ago
Senior Software Quality Engineer – AI
PeoplePilot
Toronto, ON
Full-time
We are seeking a Senior Software Quality Engineer – AI Focused to drive quality, reliability, and automation for AI-enabled software products. The role will collaborate with Development,…
Added 16 days ago
Senior Integration Engineer (Boomi/Workato)
HeartCentrix Solutions
ON · Ontario, Canada
Contract
We are seeking a Senior Integration Engineer with deep expertise in Boomi and Workato, specializing in payroll and HR system integrations. This role will focus on designing, developing, and…
Added 16 days ago
Sr. Integration Architect - Automation
Infotree Global Solutions
Toronto, ON
Contract
Job Title: Sr Integration Engineer (Intelligent Automation)
Added 16 days ago
Senior Java Software Engineer
PeoplePilot
Toronto, ON
$100,000 to $120,000 CAD / year
Full-time
Design, develop, and maintain high-quality, maintainable Java applications using clean code principles, SOLID principles, and established design patterns.
Added 16 days ago
Senior Software Engineer, Applied AI (Toronto)
Justworks
Toronto, ON
$188,000 to $242,000 CAD / year
Full-time
We are seeking a Senior Software Engineer with a strong passion for building AI-native systems in a product environment. The ideal candidate will be able to take ownership of large segments…
Added 17 days ago
Senior Quality Engineer – Test Automation, AI Testing & Java/Python
Astra-North Infoteck Inc. ~ Conquering today’s challenges, achieving tomorrow’s vision!
Toronto, ON
Full-time
Quality Engineer – Test Automation, Java/Python & GenAI
Added 17 days ago
Principal, Software Engineering – InsurTech Claims & Billing Core Systems
Intact FC
Toronto, ON · Montreal, QC
Full-time
We are looking for a Principal Engineer to provide hands-on technical leadership across Intact’s InsurTech Claims and Billing Core Systems.
Added 16 days ago
Staff/ Sr. Staff HW Test Automation Engineer
Lightmatter
Toronto, ON
As a Hardware Test Automation Engineer you'll have the opportunity to work across some combination of: an internal test software framework (Java/Python-based), industry-standard ATE test…
Added 16 days ago
Staff/ Sr. Staff HW Test Automation Engineer
Lightmatter
Toronto, ON
Full-time
As a Hardware Test Automation Engineer you'll have the opportunity to work across some combination of: an internal test software framework (Java/Python-based), industry-standard ATE test…
Added 16 days ago
Staff/ Sr. Staff HW Test Automation Engineer
Lightmatter
Toronto, ON
Full-time
As a Hardware Test Automation Engineer you'll have the opportunity to work across some combination of: an internal test software framework (Java/Python-based), industry-standard ATE test…
Added 16 days ago
Senior Engineer - Access Entitlements
Veeam Software
BC
We are seeking an exceptional Senior Engineer – Access Entitlements to design and scale the systems that discover, normalize, and reason about who has access to what across our customers'…
Added 28 days ago
Senior AI Software Engineer, Full-Stack Applications - AI Innovation Teams
Fitch Group
Toronto, ON
Senior Software Engineer, Full-Stack Applications - AI Innovation Teams
Added 17 days ago
Senior Java Engineer
K&K Global Talent Solutions INC.
Toronto, ON
Full-time
Design, develop, and maintain high-quality, maintainable Java applications using clean code principles and established design patterns.
Added 17 days ago
Staff Site Reliability Engineer
Johnson Controls
Toronto, ON · Richmond Hill, ON
Full-time
Johnson Controls is seeking a Staff Site Reliability Engineer. You will be the senior technical owner of escalated production problems, capable of debugging a failure across application…
Added 22 days ago
Senior Java Developer (with Python, AWS and Terraform)
Luxoft
Toronto, ON
Contract
We are seeking a skilled and passionate Senior Software Engineer with strong expertise in Java and AWS to join our technology team. The ideal candidate will have extensive hands-on…
Added 22 days ago
Senior Application Engineer - IAM
lululemon
Vancouver, BC
Full-time
As a Senior Application Engineer, you will lead the design and implementation of complex application solutions spanning multiple business processes and system modules, establishing…
Added 1 month ago
Staff Software Engineer - Open Source Data Platform (ODP)
Acceldata
Kitchener, ON
$180,000 to $210,000 CAD / year
Full-time
We are seeking a Staff Engineer with at least 9 years of experience in distributed systems or data platform development. In this role, you will be a technical leader responsible for…
Added 1 month ago
Principal Engineer - Open Source Data Platform (ODP)
Acceldata
Kitchener, ON
$210,000 to $240,000 CAD / year
Full-time
We are seeking a Principal Engineer with at least 14 years of experience to serve as a technical visionary and thought leader for Acceldata's Open Data Platform (ODP). In this role, you…
Added 26 days ago
Big Data Systems Engineer
Pyramid Consulting, Inc
Toronto, ON
$56 to $57.14 CAD / hour
Contract
Key Skills: SENIOR SPARK DEVELOPER, SCALA BIG DATA DEVELOPER, DATA ENGINEER, APACHE SPARK,JAVA, SPRING BOOT.
Added 26 days ago
Intermediate AI Engineer (ID#5518)
New Value Solutions
Richmond, BC
Contract
The AI Engineer will focus on hands-on development using modern AI platforms, large language models (LLMs), agentic AI frameworks, cloud technologies, APIs, and software engineering best…
Added 27 days ago
Senior Android Developer
MapsPeople
Toronto, ON
Full-time
…To support this journey, we are looking for an experienced Senior Android Developer who is comfortable leading technical initiatives, mentoring developers, and influencing engineering…
Added 29 days ago
Sr. Staff Infrastructure Engineer
VGS
Canada
Full-time
As a Senior Staff Infrastructure Engineer, you will serve as a technical leader on our Platform Engineering team. You will architect, scale, and fortify global cloud infrastructure designed…
Added 27 days ago
Software Engineer II
Planview
Vancouver, BC
Full-time
We’re seeking an Intermediate Software Engineer to join our team in Vancouver, BC. In this role, working closely with our senior team members, you will design, implement, and test features…

PeoplePilot

Senior Software Quality Engineer – AI

Toronto, ON, Canada

Job type: Full-time
Apply by: Oct 15, 2026
Posting language: English
Posted: Sep 15, 2026

Apply About employer

Applications open in a new tab.

Original job description

We are seeking a Senior Software Quality Engineer – AI Focused to drive quality, reliability, and automation for AI-enabled software products. The role will collaborate with Development, DevOps, Product, and QE teams to design robust testing strategies and scalable automation frameworks. The engineer will test AI/ML and Generative AI features, APIs, microservices, integrations, and data pipelines while ensuring reliable and secure product delivery. The position is full-time and onsite in Toronto, ON.

Key Requirements

5+ years of experience in Quality Engineering, QA, SDET, or Test Automation.
5+ years of programming experience with Java, Python, Groovy, or similar languages.
Strong experience building and maintaining test automation frameworks.
Hands-on experience with Selenium, Playwright, Cucumber/Gherkin, or RestAssured.
Experience with API, database, integration, regression, and microservices testing.
Experience with JUnit, TestNG, ReadyAPI, qTest, and JMeter.
Hands-on experience testing AI/ML, Generative AI, conversational AI, or LLM-based applications.
Knowledge of AI quality concepts such as prompt evaluation, accuracy, relevance, grounding, hallucination detection, toxicity, bias, and response consistency.
Experience with CI/CD, Agile, BDD, TDD, ATDD, and risk-based testing.
Strong analytical, problem-solving, communication, and documentation skills.

Key Responsibilities

Develop and execute comprehensive quality strategies, test plans, test cases, and test data.
Design, develop, and maintain scalable test automation frameworks.
Create automated tests for AI/ML and Generative AI features.
Evaluate AI responses for accuracy, relevance, safety, hallucination, reliability, and latency.
Test APIs, microservices, databases, integrations, data pipelines, and third-party services.
Create AI evaluation datasets, expected behaviors, quality thresholds, and regression coverage.
Integrate automated testing and AI quality checks into CI/CD pipelines.
Identify, document, prioritize, and track defects through resolution.
Analyze test coverage, quality metrics, AI evaluation results, and production feedback.
Collaborate with Development, DevOps, Product, and QE teams to improve product quality and release readiness.

Apply About employer

More about this job

Responsibilities

Develop and execute comprehensive quality strategies and scalable automation frameworks for AI-enabled software products. Evaluate AI responses for accuracy and reliability while integrating quality checks into CI/CD pipelines.

Requirements

Requires over 5 years of experience in Quality Engineering and programming with Java or Python. Must have hands-on experience testing AI/ML and Generative AI applications along with proficiency in automation tools like Selenium or Playwright.

Skills

Quality Engineering
Test Automation
Java
Python
Groovy
Selenium
Playwright
Cucumber
RestAssured
API Testing
Microservices Testing
AI/ML Testing
Generative AI
LLM Evaluation
CI/CD
Agile

Visa sponsorship

Not detected in the job text

Keywords

AI/ML
Generative AI
LLM
Prompt Evaluation
Hallucination Detection
Toxicity
Bias
JUnit
TestNG
ReadyAPI
qTest
JMeter
BDD
TDD
ATDD
Risk-based Testing
Data Pipelines
Microservices
Regression Testing
Integration Testing
Data Infrastructure
Analytics

Report this job