Quality Engineer (QE) / SDET – AI Chatbot Validation
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
Title: Quality Engineer (QE) / SDET – AI Chatbot Validation
Location : Toronto ,ON
Experience: 6+ Years
Key Responsibilities:
- Validate AI chatbot responses for accuracy, relevance, completeness, and consistency.
- Design and execute test cases, conversation scenarios, and regression suites.
- Identify hallucinations, incorrect responses, formatting issues, and knowledge gaps.
- Analyze chatbot conversations and recommend quality improvements.
- Develop and maintain automated chatbot validation scripts.
- Collaborate with AI engineers, developers, and business teams to resolve defects.
- Support UAT, release validation, and production quality monitoring.
- Track defects, test results, and quality metrics.
Required Skills:
- 5+ years of QA, SDET, and test automation experience.
- Hands-on experience testing AI, LLM, GenAI, chatbots, or virtual assistants.
- Knowledge of prompt engineering and AI response validation.
- Python, Selenium, Playwright, PyTest, or similar automation tools.
- API testing and automation frameworks.
- Jira, Confluence, GitHub, and Agile methodologies.
- Strong analytical, troubleshooting, and communication skills.
More about this job
Responsibilities
Validate chatbot responses for accuracy, relevance, completeness, and consistency; design and run test cases, conversation scenarios, and regression suites; and identify hallucinations, incorrect responses, formatting issues, and knowledge gaps. Develop automated validation scripts, collaborate with technical and business teams to resolve defects, support UAT and release validation, monitor production quality, and track test results and quality metrics.
Requirements
Requires at least five years of QA, SDET, or test automation experience, including hands-on testing of AI, LLM, generative AI, chatbots, or virtual assistants. Candidates should know prompt engineering and AI response validation, have experience with Python and automation tools such as Selenium, Playwright, or PyTest, and be familiar with API testing, Jira, Confluence, GitHub, and Agile methodologies.
Skills
- Quality Assurance
- Test Automation
- AI Chatbot Validation
- LLM Testing
- Generative AI
- Prompt Engineering
- Python
- Selenium
- Playwright
- PyTest
- API Testing
- Regression Testing
- Defect Tracking
- Analytical Skills
- Communication
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Software
- Engineering
Keywords
- Quality Engineer
- SDET
- Quality Assurance
- Test Automation
- AI Chatbot Validation
- Artificial Intelligence
- Large Language Models
- Generative AI
- Virtual Assistants
- Prompt Engineering
- AI Response Validation
- Python
- Selenium
- Playwright
- PyTest
- API Testing
- Automation Frameworks
- Test Cases
- Conversation Scenarios
- Regression Testing
- Hallucination Detection
- User Acceptance Testing
- Release Validation
- Production Quality Monitoring
- Defect Tracking
- Jira
- Confluence
- GitHub
- Agile