Quality Engineer (QE) / SDET – AI Chatbot Validation
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
Key Responsibilities:
- Design and execute functional, API, integration, regression, database, and AI-focused testing.
- Develop and maintain automation frameworks using Java, Python, or Groovy.
- Validate AI chatbot responses, prompt behavior, accuracy, hallucinations, safety, latency, and reliability.
- Test AI integrations, APIs, microservices, data pipelines, and LLM-based applications.
- Integrate automated testing into CI/CD pipelines.
- Collaborate with DevOps, Development, Product, and Agile teams to ensure high-quality releases.
Required Skills:
- 5+ years in QA/QE, test automation, or software development.
- Strong programming skills in Java, Python, or Groovy.
- Experience with Selenium, Playwright, Cucumber/Gherkin, RestAssured.
- Hands-on with JUnit, TestNG, ReadyAPI, qTest, JMeter.
- Experience testing AI/LLM applications, chatbots, conversational AI, and APIs.
- Knowledge of TDD, BDD, ATDD, continuous testing, and AI quality concepts (hallucination detection, prompt evaluation, bias, toxicity, grounding).
Preferred Skills:
- Prompt engineering, RAG, embeddings, LLM evaluation frameworks.
- Git, Bitbucket, Jenkins, SonarQube, Jira, Confluence.
- AWS, Kubernetes, Maven/Gradle.
- Performance, security, accessibility, and cloud-native testing.
- Tableau and CSTE certification.
Top 3 Must-Have Skills:
- Test automation using Java/Python/Groovy with Selenium or Playwright.
- Experience testing AI chatbots, LLMs, and Generative AI applications.
- Strong API, integration, regression testing, and CI/CD automation experience.
More about this job
Responsibilities
Design and execute functional, API, integration, regression, database, and AI-focused tests, and develop automation frameworks using Java, Python, or Groovy. Validate chatbot and LLM behavior, safety, accuracy, and reliability; integrate automated tests into CI/CD pipelines and collaborate with cross-functional Agile teams to support high-quality releases.
Requirements
Requires 5+ years of QA/QE, test automation, or software development experience, strong programming skills in Java, Python, or Groovy, and experience with test automation tools such as Selenium or Playwright. Candidates should also have experience testing AI chatbots, LLM applications, and APIs, along with knowledge of testing methodologies and AI quality concepts.
Skills
- Test Automation
- Java
- Python
- Groovy
- Selenium
- Playwright
- AI Chatbot Testing
- LLM Testing
- API Testing
- Integration Testing
- Regression Testing
- CI/CD Automation
- Prompt Evaluation
- Hallucination Detection
- JUnit
- TestNG
Visa sponsorship
Not detected in the job text
Categories
- Software
- Technology
- Engineering
Keywords
- Quality Engineering
- Test Automation
- Java
- Python
- Groovy
- Selenium
- Playwright
- Cucumber
- Gherkin
- RestAssured
- JUnit
- TestNG
- ReadyAPI
- qTest
- JMeter
- AI Chatbots
- Large Language Models
- Generative AI
- Prompt Engineering
- Retrieval-Augmented Generation
- Embeddings
- Hallucination Detection
- Bias Testing
- Toxicity Testing
- Grounding
- API Testing
- CI/CD
- Jenkins
- Kubernetes