AI Chatbot Quality Engineer
- Brampton, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
Role: AI Chatbot Quality Engineer
Location: Brampton, ON – Hybrid
Contract Role
Job Description
Key Roles Responsibilities Validate chatbot responses for accuracy| relevance| completeness| and consistency. Create and execute test scenarios| test cases| and regression test suites. Identify hallucinations| incorrect responses| formatting issues| and knowledge gaps. Perform chatbot conversation analysis and recommend quality improvements. Develop and maintain test automation for chatbot validation where possible. Work closely with developers| AI engineers| and content teams to resolve defects. Track| document| and report defects| test results| and quality metrics. Support UAT| release validation| and production quality monitoring. Key Skillsets5 Yrs of strong QA SDET Test Automation experience. Experience testing AI| LLM| GenAI| chatbots| virtual assistants| or conversational platforms. Knowledge of prompt engineering concepts Experience with Jira| Confluence| GitHub| and Agile delivery. Experience with API testing and automation frameworks. Strong analytical and troubleshooting skills. Experience with Python| Selenium| Playwright| PyTest| or similar tools. Strong communication and documentation skills.
More about this job
Responsibilities
Validate chatbot responses and analyze conversations for accuracy, relevance, completeness, consistency, hallucinations, and knowledge gaps. Create and automate test scenarios and regression suites, document and report defects and quality metrics, and support defect resolution, UAT, release validation, and production monitoring.
Requirements
Requires five years of QA, SDET, or test automation experience, including experience testing AI, LLM, generative AI, chatbot, virtual assistant, or conversational platforms. Candidates should know prompt engineering and API testing, have experience with tools such as Python, Selenium, Playwright, or PyTest and Jira, Confluence, GitHub, and Agile delivery, and demonstrate strong analytical, troubleshooting, communication, and documentation skills.
Skills
- Quality Assurance
- Test Automation
- AI Testing
- LLM Testing
- Generative AI
- Chatbot Testing
- Prompt Engineering
- API Testing
- Python
- Selenium
- Playwright
- PyTest
- Jira
- Confluence
- GitHub
- Agile
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Software
- Engineering
Keywords
- Quality Assurance
- SDET
- Test Automation
- AI
- LLM
- Generative AI
- Chatbots
- Virtual Assistants
- Conversational Platforms
- Prompt Engineering
- Test Scenarios
- Test Cases
- Regression Testing
- Chatbot Conversation Analysis
- Hallucination Detection
- API Testing
- Python
- Selenium
- Playwright
- PyTest
- Jira
- Confluence
- GitHub
- Agile
- User Acceptance Testing
- Release Validation
- Production Quality Monitoring
- Defect Tracking
- Quality Metrics