Quality Engineer (QE) / SDET – AI Chatbot Validation

Job Title: Quality Engineer (QE) / SDET – AI Chatbot Validation

Locations: ~Brampton, Ontario~Scarborough, Ontario~Etobicoke, Ontario~Guleph, Ontario~Hamilton, Ontario~Kitchener, Ontario~Milton, Ontario~North York, Ontario~Ontario Nes,~Oshawa, Ontario~

Long Term Contract

Experience: 6–8 Years

Work Model: Hybrid – 1 day onsite per week

Job Description:

We are looking for an experienced Quality Engineer / SDET with strong test automation skills and hands-on experience validating AI-powered chatbots, LLMs, Generative AI applications, or conversational platforms. The ideal candidate will ensure the accuracy, relevance, consistency, and reliability of AI-generated responses while developing and maintaining automated testing solutions.

Key Responsibilities:

• Validate chatbot responses for accuracy, relevance, completeness, and consistency.
• Design, develop, and execute test scenarios, test cases, and regression test suites.
• Identify AI hallucinations, incorrect responses, formatting issues, and knowledge gaps.
• Analyze chatbot conversations and recommend quality improvements.
• Develop and maintain automated test frameworks for chatbot validation.
• Collaborate with developers, AI engineers, and content teams to investigate and resolve defects.
• Track defects, document test results, and report quality metrics.
• Support UAT, release validation, and production quality monitoring.

Required Skills:

• 5+ years of strong QA / SDET / test automation experience.
• Experience testing AI, LLMs, Generative AI applications, chatbots, virtual assistants, or conversational platforms.
• Understanding of prompt engineering and AI response validation.
• Strong programming and automation skills using Python, Selenium, Playwright, PyTest, or similar tools.
• Experience with API testing and automation frameworks.
• Familiarity with Jira, Confluence, GitHub, and Agile delivery methodologies.
• Strong analytical, troubleshooting, communication, and documentation skills.

Preferred Skills:

• Experience with LLM evaluation, hallucination detection, and conversational AI quality metrics.
• Experience building automated validation and regression testing frameworks for AI applications.
• Knowledge of Generative AI quality engineering practices.