Quality Engineer (QE) / SDET – AI Chatbot Validation
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
Job Title: Quality Engineer (QE) / SDET – AI Chatbot Validation
Locations: ~Brampton, Ontario~Scarborough, Ontario~Etobicoke, Ontario~Guleph, Ontario~Hamilton, Ontario~Kitchener, Ontario~Milton, Ontario~North York, Ontario~Ontario Nes,~Oshawa, Ontario~
Long Term Contract
Experience: 6–8 Years
Work Model: Hybrid – 1 day onsite per week
Job Description:
We are looking for an experienced Quality Engineer / SDET with strong test automation skills and hands-on experience validating AI-powered chatbots, LLMs, Generative AI applications, or conversational platforms. The ideal candidate will ensure the accuracy, relevance, consistency, and reliability of AI-generated responses while developing and maintaining automated testing solutions.
Key Responsibilities:
- Validate chatbot responses for accuracy, relevance, completeness, and consistency.
- Design, develop, and execute test scenarios, test cases, and regression test suites.
- Identify AI hallucinations, incorrect responses, formatting issues, and knowledge gaps.
- Analyze chatbot conversations and recommend quality improvements.
- Develop and maintain automated test frameworks for chatbot validation.
- Collaborate with developers, AI engineers, and content teams to investigate and resolve defects.
- Track defects, document test results, and report quality metrics.
- Support UAT, release validation, and production quality monitoring.
Required Skills:
- 5+ years of strong QA / SDET / test automation experience.
- Experience testing AI, LLMs, Generative AI applications, chatbots, virtual assistants, or conversational platforms.
- Understanding of prompt engineering and AI response validation.
- Strong programming and automation skills using Python, Selenium, Playwright, PyTest, or similar tools.
- Experience with API testing and automation frameworks.
- Familiarity with Jira, Confluence, GitHub, and Agile delivery methodologies.
- Strong analytical, troubleshooting, communication, and documentation skills.
Preferred Skills:
- Experience with LLM evaluation, hallucination detection, and conversational AI quality metrics.
- Experience building automated validation and regression testing frameworks for AI applications.
- Knowledge of Generative AI quality engineering practices.
More about this job
Responsibilities
Validate AI chatbot responses for accuracy, relevance, completeness, and consistency, and identify hallucinations, formatting issues, and knowledge gaps. Build and maintain automated validation and regression frameworks, investigate defects with cross-functional teams, and support UAT, release validation, and production quality monitoring.
Requirements
Requires at least five years of QA, SDET, or test automation experience, including hands-on testing of AI, LLM, Generative AI, chatbot, or conversational applications. Candidates should have strong programming and automation skills, experience with API testing, familiarity with prompt engineering and tools such as Python, Selenium, Playwright, PyTest, Jira, Confluence, and GitHub, and strong analytical and communication skills.
Skills
- Quality Assurance
- Software Development Test Engineering
- Test Automation
- AI Chatbot Validation
- Large Language Models
- Generative AI
- Prompt Engineering
- Python
- Selenium
- Playwright
- PyTest
- API Testing
- Hallucination Detection
- Regression Testing
- Jira
- Agile
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Software
- Engineering
Keywords
- Quality Engineer
- SDET
- Quality Assurance
- Test Automation
- AI Chatbot Validation
- Large Language Models
- Generative AI
- Conversational Platforms
- Virtual Assistants
- Prompt Engineering
- AI Response Validation
- Hallucination Detection
- Python
- Selenium
- Playwright
- PyTest
- API Testing
- Automated Testing Frameworks
- Regression Testing
- UAT
- Release Validation
- Production Quality Monitoring
- Jira
- Confluence
- GitHub
- Agile
- Defect Tracking
- LLM Evaluation
- AI Quality Metrics