Jobs in Canada

Group jobs by employerEmail me new jobs Save this search

Added 1 day ago
XG
DevOps Engineer , Mississauga, ON
Xforia Global Talent & Technology Solutions
Mississauga, ON
- Contract
Skills: Distributed Systems development, Java, C/C++, Python, JavaScript (Node.js), Linux, CI/CD, GitHub, AI assisted development workflows.
Added 1 day ago
KG
Kafka Engineer
K&K Global Talent Solutions INC.
Montreal, QC
- Full-time
We are looking for a Kafka Engineer with strong experience in Apache Kafka, Python, Linux, and Grafana to support and enhance large-scale distributed systems within an Investment Banking…
Added 1 day ago
AI
SAS / Python Developer
Apptoza Inc.
Toronto, ON
- Contract
Develop, test, and maintain high-quality Python applications, scripts, and automation solutions.
Added 1 day ago
FB
Junior Data Analys
F&B Business News
Canada
- Full-time
…early-career professionals who are passionate about working with data, identifying trends, and providing insights that support business…
Added 1 day ago
QC
Full-Stack Developer (T4 employee)
Quarry Consulting
Brampton, ON
- Contract
*** MUST have been in Canada for 5+ years and ONLY open to Term employees*** · Angular (must have)
Added 1 day ago
C
System Integrator (Renewable)
CIMA+
Guelph, ON
- $67,440 to $92,730 CAD / year
- Full-time
Experience with SCADA/HMI programming on platforms such as Inductive Automation Ignition (Perspective & Vision), Wonderware InTouch, GE Proficy iFix, VTscada.
Added 1 day ago
B
AI Quality Engineer / AI Test Automation Engineer
BullTechSystem
Toronto, ON
- Full-time
Design, develop, and maintain scalable test automation frameworks using Java, Python, or Groovy.
Added 1 day ago
XG
DevSecOps Engineer
Xforia Global Talent & Technology Solutions
Mississauga, ON
- Contract
Collaborate with engineering teams across multiple languages (Java, C/C++, Python, Node.js).
Added 1 day ago
I-
Systems Testing/QA Specialist - Intermediate - 859
I8IS - Infiniti Software Solutions
Toronto, ON
- Full-time
· Understanding of scripting languages (e.g., Python, JavaScript, Bash) for automation or test data setup.
Added 1 day ago
Technology - Brokerage Engineering - Senior Data Engineer
Questrade Financial Group
Toronto, ON
Work-life balance in a hybrid environment with at least 3 days in office
Added 1 day ago
C
Senior DevOps Specialist
CIRA
Ottawa, ON
- Full-time
As a Senior DevOps Specialist, you will collaborate with development, security and product teams to build and maintain critical internet infrastructure. You are passionate about designing…
Added 1 day ago
BQ
QNX DevOps Developer
BlackBerry QNX
Ottawa, ON
- Full-time
With the evolving landscape of safety (ISO2626) and security (ISO21434) required standards, QNX is in need to increase its developer productivity by scaling internal processes to benefit…
Added 1 day ago
B
QNX DevOps Developer
BlackBerry
Ottawa, ON
- Full-time
With the evolving landscape of safety (ISO2626) and security (ISO21434) required standards, QNX is in need to increase its developer productivity by scaling internal processes to benefit…
Added 1 day ago
SM
RQ11860 – Database Administrator – Senior
S M Software Solutions Inc
Toronto, ON
- Internship
Work Arrangement: Onsite – 5 Days per Week
Added 1 day ago
SM
RQ11859 – Systems Testing/QA Specialist – Intermediate
S M Software Solutions Inc
Toronto, ON
- Internship
Work Arrangement: Onsite – 5 Days per Week
Added 1 day ago
C
Python Engineer
Collabera
Toronto, ON
- $85 to $95 CAD / hour
- Contract
Seeking a senior, hands-on Python Software Engineer to support the Total Fund Management Portfolio Management Technology team. The ideal candidate is a self-directed engineer with…
Added 1 day ago
LS
Ingénieur(e) - Équipements et intégration de procédés
Lock Search Group
Montreal, QC · Greater Montreal Metropolitan Area
- $80,000 to $110,000 CAD / year
- Full-time
Familiarité avec les outils CAO (SolidWorks, AutoCAD ou équivalent), les P&ID et la programmation (notamment Python).
Added 1 day ago
SS
Ingénieur SIEM / SIEM Engineer
Sky Systems, Inc. (SkySys)
Montreal, QC
- $90 to $95 CAD / hour
- Contract
The candidate must possess the following qualifications:
Added 1 day ago
TC
Python SAS
Tata Consultancy Services
Toronto, ON
- $100,000 to $120,000 CAD / year
- Full-time
Develop, test, and maintain high-quality Python applications, scripts, and automation solutions.
Added 1 day ago
SN
Integrations Developer
Six Nations Polytechnic
Brantford, ON
- Contract
Over 3 years and including 5 years current related professional experience as an Integrations Developer or backend software developer. Requires hands-on technical experience with core…
Added 2 days ago
T
Senior Azure DevOps Engineer
Teqfocus
Toronto, ON
- Contract
We are looking for an experienced Senior Azure DevOps & Infrastructure Engineer with strong expertise in CI/CD pipeline development, Microsoft Azure cloud infrastructure, Infrastructure as…
Added 2 days ago
T
Senior Full Stack AI Developer
Teqfocus
Toronto, ON
- Contract
We are looking for an experienced Senior Full Stack AI Developer with strong expertise in Python, modern full stack development, and Artificial Intelligence (AI) to join our team in Toronto.
Added 2 days ago
N
POS Integration & Implementation Engineer – Retail Data
nugget.ai
Mississauga, ON
- Contract
Develop Python scripts for data processing, automation, and integration workflows.
Added 2 days ago
GC
Ingénieur(e) sénior SDET / Senior Software Development Engineer in Test
Game Companies
Montreal, QC
- Full-time
Expérience avec Python, particulièrement dans le cadre d’initiatives d’intelligence artificielle et d’automatisation.
Added 2 days ago
GC
QA Technique Senior / Senior Technical QA
Game Companies
Montreal, QC
- Full-time
Develop and maintain test automation scripts in Python.

BullTechSystem

AI Quality Engineer / AI Test Automation Engineer

Toronto, ON, Canada

Full-time

Apply on LinkedIn About employer

Applications open in a new tab.

Job details

Job type: Full-time
Apply by: Dec 11, 2026
Posting language: English
Posted: Nov 11, 2026

Original job description

Roles and Responsibilities:

Partner with DevOps, Development, Product, and QE teams to design and execute exploratory, automated, and AI-focused tests that ensure product quality.

Define quality strategies, test plans, test cases, and test data for functional, integration, system, regression, API, database, and acceptance testing.

Design, develop, and maintain scalable test automation frameworks using Java, Python, or Groovy.

Build automated tests for AI-enabled features, including prompt behavior, response accuracy, hallucination risks, safety, reliability, latency, and regression coverage.

Validate AI integrations, APIs, microservices, data pipelines, model interactions, and third-party services.

Establish testing approaches for both deterministic and non-deterministic AI behavior, including evaluation datasets, expected behavior, quality thresholds, and repeatability.

Integrate automated testing and AI quality checks into CI/CD pipelines in partnership with DevOps teams.

Use tools and frameworks such as Selenium, Cucumber/Gherkin, Playwright, RestAssured, JUnit, TestNG, ReadyAPI, and JMeter.

Identify, document, prioritize, and track defects through resolution.

Analyze test coverage, quality trends, AI evaluation results, and production feedback to identify gaps and drive continuous improvement.

Collaborate with cross-functional Agile teams to deliver reliable, secure, and user-focused product features.

Communicate test results, risks, quality metrics, and release recommendations clearly to technical and non-technical stakeholders.

Required Skills:

5+ years of relevant experience in Quality Engineering, Quality Assurance, test automation, or software development.

5+ years of programming experience with Java, Python, Groovy, or a comparable language.

5+ years of experience building and maintaining test automation frameworks.

Hands-on experience with Selenium, Cucumber/Gherkin, Playwright, RestAssured, or similar automation technologies.

Experience testing functional and non-functional requirements, APIs, integrations, databases, microservices, and regression suites.

Experience with testing frameworks and tools such as JUnit, TestNG, qTest, ReadyAPI, and JMeter.

Understanding of TDD, ATDD, BDD, continuous testing, and risk-based testing practices.

Experience testing AI or machine learning-powered applications, APIs, conversational systems, or generative AI features.

Understanding of AI quality concepts such as prompt evaluation, grounding, accuracy, relevance, toxicity, bias, hallucination detection, and response consistency.

Ability to create effective test data and evaluation datasets for AI-driven functionality.

Strong analytical, problem-solving, communication, and technical documentation skills.

Ability to work effectively in a fast-paced, cross-functional Agile environment.

Nice to Have:

Experience with AI evaluation frameworks, LLM testing, prompt engineering, retrieval-augmented generation, embeddings, or model observability.

Experience with Git, Bitbucket, Jenkins, CI/CD pipelines, Artifactory, SonarQube, Jira, Confluence, AWS, Kubernetes, Maven, or Gradle.

Experience testing cloud-native applications and distributed microservice architectures.

Experience with security, performance, accessibility, or reliability testing.

Experience developing automation scripts and reusable testing utilities.

Experience with Tableau or other analytics and visualization platforms.

CSTE certification or an equivalent quality engineering certification.

Experience working directly with DevOps teams to implement automated quality gates and release controls.

Apply on LinkedIn About employer

More about this job

Responsibilities

Design and execute functional, integration, API, regression, and AI-focused tests, while building scalable automation frameworks and integrating quality checks into CI/CD pipelines. Evaluate AI behavior for accuracy, safety, reliability, consistency, and performance; track defects, analyze quality results, and communicate risks and release recommendations to stakeholders.

Requirements

Requires at least five years of relevant quality engineering or software development experience, programming experience, and experience building test automation frameworks. Candidates should have hands-on automation and API testing experience, knowledge of testing methodologies, and experience evaluating AI or machine learning applications and their quality characteristics.

Skills

Quality Engineering
Test Automation
Java
Python
Groovy
AI Testing
Machine Learning Testing
Selenium
Playwright
Cucumber/Gherkin
API Testing
CI/CD
Test Strategy
Risk-Based Testing
Agile
AI Evaluation

Visa sponsorship

Not detected in the job text

Keywords

Quality Engineering
Quality Assurance
Test Automation
Java
Python
Groovy
Selenium
Cucumber/Gherkin
Playwright
RestAssured
JUnit
TestNG
ReadyAPI
JMeter
qTest
AI Testing
Machine Learning
Generative AI
Prompt Evaluation
Hallucination Detection
Retrieval-Augmented Generation
CI/CD
Jenkins
AWS
Kubernetes
TDD
ATDD
BDD
Risk-Based Testing
Agile

Report this job

Work requirements

Jobs in Canada