Jobs in Canada

Group jobs by employerEmail me new jobs Save this search

Added 2 months ago
C
Senior Test Automation Engineer – AI Testing
Cognizant
Montreal, QC
- Full-time
…langage (LLM), notamment de la tokénisation, des vecteurs d'intégration (embeddings), des mécanismes d'attention et de l'inférence, afin d'anticiper les modes de défaillance…
Added 7 months ago
S
Technical Engineer, Healthcare IT & PACS/Spécialiste technique, TI en santé et PACS
Sectra
Montreal, Quebec
- Full-time
…service-minded individuals with a passion for healthcare. As a Technical Engineer your role will focus on designing, deploying, and supporting critical systems that drive improved clinical…
Added 7 months ago
S
Technical Engineer, Healthcare IT & PACS/Spécialiste technique, TI en santé et PACS
Sectra
Mississauga, Ontario
- Full-time
…service-minded individuals with a passion for healthcare. As a Technical Engineer your role will focus on designing, deploying, and supporting critical systems that drive improved clinical…
Added 7 months ago
SV
Principal Engineer - Stackpoint Ventures
Stackpoint Ventures
Remote, Canada
- Full-time
- Remote
As a Principal Engineer, you are the person who builds. You'll typically embed with a single portfolio company for several months at a time - architecting, coding, and shipping backend…
Added 7 months ago
PE
Senior HVDC Construction Engineer
Pattern Energy
Canada
- Full-time
Work closely with project managers, design engineers, Owner's Engineer, Contractors, equipment vendor & suppliers and Engineer's of other Stakeholder during project planning and execution…
Added 4 months ago
FA
Software Engineer - Data
Forum Asset Management
Toronto, Ontario
- Full-time
Forum Asset Management – Software Engineer - Data
Added 4 months ago
S
OptoMechanical Engineer
SBQuantum
Sherbrooke, Quebec
- Full-time
As Optomechanical Engineer, you will own the evolution of SBQuantum's diamond quantum magnetometer optomechanical subsystem. You will design, fabricate, test, and industrialize the…
Added 2 months ago
MR
Engineer Computer - Cosmopolitan
MGM Resorts International
NB · Chiasson Office, NB
- Full-time
The Deployment Engineer will be responsible for generating new and innovative solutions to complex problems and propose improvements to processes.
Added 2 months ago
H
Senior Low-Level Control Engineer
Humanoid
Burnaby, BC
- Full-time
We are looking for a Senior Control Engineer to join our Controls Team in Vancouver, focusing on SW control development and integration of robotic actuators and sensors.
Added 2 months ago
GT
Director, Technology Operations
Goway Travel
Toronto, ON
- Full-time
Monitoring of integration paths, error and failure-rate thresholds, and defined fallback behaviour when a partner degrades.
Added 2 months ago
T
Directeur(trice), solutions clients SAP | SAP Customer Solution Director
Talan
Montreal, QC
- Full-time
Définir les hypothèses de solution, les limites de portée, les exclusions et les intrants fonctionnels aux estimations, puis challenger les estimations avant leur intégration au prix.
Added 2 months ago
D
Senior Product Engineer I - Automation
Docebo
Remote, Canada
- $102,800 to $137,000 CAD / year
- Full-time
- Remote
…will architect, build, and scale business-critical backend systems in Go and PHP that seamlessly onboard millions of users. You'll serve as the primary technical advisor and force…
Added 7 months ago
K
Electrical Engineer
Kollabtek
Laval, Quebec
- Full-time
Kollabtek is looking for an Electrical Engineer to support our customers in the development and integration of embedded electrical systems. In this role, you will be responsible for the…
Added 4 months ago
Senior Engineer (Hybrid)
Kerr Wood Leidal Associates
Calgary, Alberta · Burnaby, British Columbia
Evaluate infrastructure alternatives and support integration of water management systems into mine plans.
Added 3 months ago
AT
Staff Embedded Firmware Engineer (Contractor)
Arlo Technologies
Richmond, British Columbia · CA - British Columbia - Richmond
- Full-time
The ideal candidate has deep expertise in embedded systems, video streaming technologies, and real-time software development, along with a proven track record of leading complex firmware…
Added 6 months ago
OI
Senior Hardware Engineer
Ottawa Infotainment
Ottawa, Ontario · Kanata, Ontario
- Full-time
…digital cockpit through high-performance compute, embedded software, safety-critical systems, and AI-driven…
Added 6 months ago
OI
Senior Embedded Software Engineer
Ottawa Infotainment
Ottawa, Ontario · Kanata, Ontario
- Full-time
…digital cockpit through high-performance compute, embedded software, safety-critical systems, and AI-driven…
Added 4 months ago
AT
Senior Embedded Firmware Engineer (Contractor)
Arlo Technologies
Richmond, British Columbia · CA - British Columbia - Richmond
- Full-time
The ideal candidate has strong experience in embedded systems, video streaming technologies, and real-time software development, with the ability to independently own complex firmware…
Added 2 months ago
E
Founding Fullstack Engineer
Eklan
Toronto, ON
- $61,000 to $142,000 CAD / year
- Full-time
This is not a narrow model-integration role. You will influence our architecture, product, AI systems, engineering culture and technical strategy. You will move between understanding…
Added 2 months ago
MT
Physical Design Engineer
Mirafra Technologies
ON · Ontario, Canada
- Full-time
We are seeking a Physical Design Engineer with a strong foundation in Static Timing Analysis (STA) and a focus on design flow automation. In this role, you will be responsible for driving…
Added 2 months ago
AE
Technician Programmer
Associated Engineering
Saskatoon, SK
- Full-time
…active in the delivery of servicing to power and automation systems with instrumentation to monitor medium voltage electrical service, process, metering, primary and secondary switchgear,…
Added 2 months ago
Electro-Optical Engineering Co-Op (Waterdown, CAN)
L3Harris Technologies
Hamilton, ON · Waterdown, ON
As a subsidiary of L3Harris Technologies and within the Integrated Mission Systems Segment, the WESCAM division develops the most advanced optical sensor and targeting systems for global…
Added 2 months ago
AC
Electrical Engineer-Power Metering /Protection Relays / Instrume
Accuenergy Canada Inc
Toronto, ON
- $55,000 to $70,000 CAD / year
- Full-time
…and electronic communications, instrumentation and control systems, equipment, and…
Added 4 months ago
TH
Senior AI Engineer - Full time
Trillium Health Partners
Mississauga, Ontario
- Full-time
Reporting to the Director, AI & Digital Planning, the Senior AI Engineer will be responsible for using the latest AI tools to design, prototype, build, evaluate, and scale AI solutions…
Added 4 months ago
DS
Backend Engineer
Devoted Studios
Remote, Canada
- Remote
We are looking for a Backend Engineer with strong Java and C++ experience to work on backend systems supporting game functionality and the client/server interface.

Cognizant

Senior Test Automation Engineer – AI Testing

Montreal, QC, Canada

Full-time

Apply on LinkedIn About employer

Applications open in a new tab.

Job details

Job type: Full-time
Posting language: English · French
Posted: Sep 2, 2026

Original job description

English section of this bilingual listing.

English Version

About The Role

We are looking to hire a Senior QA Engineer with AI experience, automation, and performance testing. Automation will be Python, Robot Framework, Playwright.

In this role, you will

AI Systems and Models Testing

Design and execute comprehensive test strategies for AI systems and models, including prompt engineering, output evaluation, and bias/safety testing.

Develop deep understanding of LLM behavior - tokenization, embeddings, attention mechanisms, and inference - to anticipate failure modes.

Construct effective prompts, recognize hallucinations and off-target outputs, and assess quality across accuracy, tone, coherence, and bias dimensions.

Apply evaluation metrics specific to generative AI and establish appropriate thresholds.

Test AI systems integrated with RAG pipelines and knowledge bases, validating data quality and retrieval accuracy as they impact model outputs.

Understand vector database mechanics, similarity search thresholds, embedding drift, and test edge cases including near-duplicate documents, sparse vs. dense embeddings, and performance under scale.

Leverage LangChain and LangGraph frameworks to read code, understand chain and graph construction, identify failure points, and write test harnesses.

Validate integration points using MCPs, testing tool availability and error handling.

Test Strategy and Planning

Define and execute comprehensive test strategies for securitization platforms, ensuring coverage across functional, regression, integration, and performance testing. Establish testing standards and best practices that span both traditional QA and AI-specific validation.

Test Automation and Framework Development

Design, build, and maintain automated test suites to accelerate release cycles and improve coverage. Leverage AI and ML tools to enhance test coverage, improve efficiency, and reduce regression cycles.

Securitization Lifecycle QA Validate end-to-end deal workflows including setup, structuring, processing, and distributions. Ensure data integrity across upstream and downstream systems through reconciliation testing and reporting.

Release and Regression Testing

Coordinate regression testing for platform releases, patches, and infrastructure changes. Ensure stability and backward compatibility, particularly during critical processing windows.

Work model

We believe hybrid work is the way forward as we strive to provide flexibility wherever possible. Based on this role’s business requirements, this is a hybrid position requiring 4 days a week in a client or Cognizant office in Montreal, QC. Regardless of your working arrangement, we are here to support a healthy work-life balance though our various wellbeing programs.

The working arrangements for this role are accurate as of the date of posting. This may change based on the project you’re engaged in, as well as business and client requirements. Rest assured; we will always be clear about role expectations.

What You Need To Have To Be Considered

7+ years of quality assurance or quality engineering experience.

Technical Skills

Hands-on experience with test automation tools (Selenium, Robot Framework, Playwright, or similar). Proficiency in programming languages including Java and Python, with demonstrated framework implementation expertise. Hands-on API automation and backend system validation experience. Proficiency in database query development, data validation, and reconciliation testing. Experience with CI/CD pipelines and DevOps practices (Jenkins, GitHub, or similar).

AI and ML Competencies

Core understanding of LLM architecture and behavior. Hands-on experience with LangChain and/or LangGraph frameworks. Knowledge of RAG pipelines, vector databases, and agentic solutions. Familiarity with Model Context Protocols (MCPs) and integration testing. Understanding of bias, safety, and red-team testing methodologies for AI systems.

We're excited to meet people who share our mission and can make an impact in a variety of ways. Don't hesitate to apply, even if you only meet the minimum requirements listed. Think about your transferable experiences and unique skills that make you stand out as someone who can bring new and exciting things to this role.

Read the full bilingual original

À propos du poste

Nous recherchons un Ingénieur en Assurance Qualité, ayant de l'expérience en IA et en tests d'automatisation et de performance. Les outils d'automatisation des tests utilisés sont Python, Robot Framework, Playwright.

Ce que vous accomplirez

Tests de systèmes et modèles d’IA

Concevoir et exécuter des stratégies de test complètes pour les systèmes et modèles d’IA, incluant l'ingénierie des requêtes (prompt engineering), l’évaluation des résultats et les tests de biais/sécurité.
Développer une compréhension approfondie du comportement des grands modèles de langage (LLM), notamment de la tokénisation, des vecteurs d'intégration (embeddings), des mécanismes d'attention et de l'inférence, afin d'anticiper les modes de défaillance potentiels.
Concevoir des requêtes efficaces, détecter les hallucinations et les résultats hors cible, et évaluer la qualité des réponses générées selon leur exactitude, leur ton, leur cohérence et l'absence de biais.
Appliquer des mesures d'évaluation propres à l'IA générative et établir des seuils de performance appropriés.
Tester les systèmes d'IA intégrés à des pipelines de génération augmentée par récupération (RAG) et à des bases de connaissances, en veillant à la qualité des données et à l'exactitude de la récupération de l'information, lesquelles ont une incidence sur les résultats générés par les modèles.
Comprendre le fonctionnement des bases de données vectorielles, les seuils de recherche par similarité, les phénomènes de dérive des embeddings, et tester les cas limites, notamment les documents quasi identiques, les embeddings clairsemés ou denses, ainsi que les performances lors du passage à l'échelle.
Utiliser les frameworks LangChain et LangGraph afin d'analyser le code, de comprendre la structure des chaînes et des graphes, d'identifier les points de défaillance et de concevoir des harnais de test.
Valider les points d'intégration à l'aide des protocoles MCP (Model Context Protocol), en vérifiant la disponibilité des outils et la gestion des erreurs.

Stratégie et planification des tests

Élaborer et exécuter des stratégies de test complètes pour les plateformes de titrisation, en veillant à la couverture complète des tests fonctionnels, de régression, d'intégration et de performance. Établir des normes d'essai et des pratiques exemplaires couvrant à la fois l'assurance qualité traditionnelle et la validation propre aux systèmes d'IA.

Automatisation des tests et développement de cadres d'automatisation

Concevoir, développer et maintenir des suites de tests automatisés afin d'accélérer les cycles de mise en production et d'améliorer la couverture des essais. Mettre à profit les outils d'IA et d'apprentissage automatique (ML) pour accroître la couverture des essais, améliorer l'efficacité des activités de test et réduire les cycles de régression.

Assurance qualité du cycle de vie de la titrisation Valider de bout en bout les flux de travail liés aux transactions, notamment la configuration, la structuration, le traitement et les distributions. Assurer l'intégrité des données entre les systèmes en amont et en aval au moyen de tests de rapprochement et de rapports.

Tests de mise en production et de régression

Coordonner les tests de régression liés aux mises en production de la plateforme, aux correctifs et aux modifications d'infrastructure. Assurer la stabilité et la rétrocompatibilité, en particulier lors des périodes de traitement critiques.

Work model

Nous croyons que le travail hybride est l’avenir, et nous nous efforçons d'offrir de la flexibilité dans la mesure du possible. Compte tenu des exigences opérationnelles de ce poste, il s'agit d'un poste hybride nécessitant quatre jours par semaine au bureau de Cognizant ou chez un client à Montreal, QC. Quel que soit votre modalité de travail, nous sommes là pour favoriser une équilibre travail-vie personnelle grâce à nos programmes de mieux-être.

Les modalités de travail pour ce poste sont exactes à la date de publication. Elles peuvent changer en fonction du projet dans lequel vous êtes engagé, ainsi que des exigences de Cognizant et du client. Soyez assuré(e) que nous serons toujours clairs sur les attentes liées au poste.

Compétences requises

Plus de 7 ans d'expérience en assurance qualité ou en ingénierie de la qualité. Solide connaissance du domaine de la titrisation, des marchés des capitaux ou de catégories d’actifs similaires.

Compétences techniques

Expérience pratique avec des outils d'automatisation des tests tels que Selenium, Robot Framework, Playwright ou des outils comparables. Maîtrise des langages de programmation, notamment Java et Python, ainsi qu’une expertise avérée dans la mise en œuvre de cadres d’automatisation. Expérience concrète en automatisation des tests d’API et en validation des systèmes back-end. Maîtrise du développement de requêtes de bases de données, de la validation des données et des tests de rapprochement. Expérience des pipelines CI/CD et des pratiques DevOps, notamment avec Jenkins, GitHub ou des outils comparables.

Compétences en IA et en apprentissage automatique

Bonne compréhension de l’architecture et du fonctionnement des grands modèles de langage (LLM). Expérience pratique avec les frameworks LangChain et/ou LangGraph. Connaissance des pipelines RAG, des bases de données vectorielles et des solutions agentiques. Connaissance des protocoles MCP (Model Context Protocol) et expérience des tests d'intégration. Compréhension des méthodes d’évaluation des biais, de la sécurité et des exercices de simulation d’attaque (red teaming) appliqués aux systèmes d’IA.

Nous avons hâte de rencontrer des personnes qui partagent notre mission et qui souhaitent contribuer de différentes façons. N’hésitez pas à postuler, même si vous ne répondez qu’aux compétences requises. Vos expériences et vos compétences transférables comptent - montrez-nous comment vous pourriez être la personne qu’il nous faut pour ce poste.

English Version

About The Role

We are looking to hire a Senior QA Engineer with AI experience, automation, and performance testing. Automation will be Python, Robot Framework, Playwright.

In this role, you will

AI Systems and Models Testing

Design and execute comprehensive test strategies for AI systems and models, including prompt engineering, output evaluation, and bias/safety testing.

Develop deep understanding of LLM behavior - tokenization, embeddings, attention mechanisms, and inference - to anticipate failure modes.

Construct effective prompts, recognize hallucinations and off-target outputs, and assess quality across accuracy, tone, coherence, and bias dimensions.

Apply evaluation metrics specific to generative AI and establish appropriate thresholds.

Test AI systems integrated with RAG pipelines and knowledge bases, validating data quality and retrieval accuracy as they impact model outputs.

Understand vector database mechanics, similarity search thresholds, embedding drift, and test edge cases including near-duplicate documents, sparse vs. dense embeddings, and performance under scale.

Leverage LangChain and LangGraph frameworks to read code, understand chain and graph construction, identify failure points, and write test harnesses.

Validate integration points using MCPs, testing tool availability and error handling.

Test Strategy and Planning

Define and execute comprehensive test strategies for securitization platforms, ensuring coverage across functional, regression, integration, and performance testing. Establish testing standards and best practices that span both traditional QA and AI-specific validation.

Test Automation and Framework Development

Design, build, and maintain automated test suites to accelerate release cycles and improve coverage. Leverage AI and ML tools to enhance test coverage, improve efficiency, and reduce regression cycles.

Securitization Lifecycle QA Validate end-to-end deal workflows including setup, structuring, processing, and distributions. Ensure data integrity across upstream and downstream systems through reconciliation testing and reporting.

Release and Regression Testing

Coordinate regression testing for platform releases, patches, and infrastructure changes. Ensure stability and backward compatibility, particularly during critical processing windows.

Work model

We believe hybrid work is the way forward as we strive to provide flexibility wherever possible. Based on this role’s business requirements, this is a hybrid position requiring 4 days a week in a client or Cognizant office in Montreal, QC. Regardless of your working arrangement, we are here to support a healthy work-life balance though our various wellbeing programs.

The working arrangements for this role are accurate as of the date of posting. This may change based on the project you’re engaged in, as well as business and client requirements. Rest assured; we will always be clear about role expectations.

What You Need To Have To Be Considered

7+ years of quality assurance or quality engineering experience.

Technical Skills

Hands-on experience with test automation tools (Selenium, Robot Framework, Playwright, or similar). Proficiency in programming languages including Java and Python, with demonstrated framework implementation expertise. Hands-on API automation and backend system validation experience. Proficiency in database query development, data validation, and reconciliation testing. Experience with CI/CD pipelines and DevOps practices (Jenkins, GitHub, or similar).

AI and ML Competencies

Core understanding of LLM architecture and behavior. Hands-on experience with LangChain and/or LangGraph frameworks. Knowledge of RAG pipelines, vector databases, and agentic solutions. Familiarity with Model Context Protocols (MCPs) and integration testing. Understanding of bias, safety, and red-team testing methodologies for AI systems.

We're excited to meet people who share our mission and can make an impact in a variety of ways. Don't hesitate to apply, even if you only meet the minimum requirements listed. Think about your transferable experiences and unique skills that make you stand out as someone who can bring new and exciting things to this role.

Apply on LinkedIn About employer

More about this job

Responsibilities

Design and execute comprehensive test strategies for AI systems and LLMs, focusing on prompt engineering, RAG pipelines, and bias detection. Develop and maintain automated test suites for securitization platforms to ensure data integrity and system stability.

Requirements

Requires over 7 years of QA experience with proficiency in Python, Java, and automation tools like Playwright or Robot Framework. Must have a deep understanding of LLM behavior, vector databases, and experience in capital markets or securitization.

Skills

AI Testing
Python
Robot Framework
Playwright
LLM Architecture
Prompt Engineering
RAG Pipelines
Vector Databases
LangChain
LangGraph
API Automation
Java
CI/CD
Securitization
Red Teaming
Model Context Protocol

Benefits

Wellbeing Programs

Visa sponsorship

Not detected in the job text

Keywords

AI
Machine Learning
LLM
RAG
Vector Database
LangChain
LangGraph
Python
Java
Robot Framework
Playwright
Selenium
API Automation
CI/CD
Jenkins
GitHub
Securitization
Capital Markets
Prompt Engineering
Red Teaming
Model Context Protocol
Tokenization
Embeddings
Regression Testing
Performance Testing
DevOps
Data Validation
Reconciliation Testing

Report this job

Work requirements

Jobs in Canada