AI/Quality Engineer
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Experience
- Minimum 2+ years of hands-on experience in AI/ML testing, Generative AI evaluation, or LLM quality assessment.
- Posting language
- English
- Posted
Original job description
Position: AI/Quality Engineer
Location: Toronto, ON (2-3 days hybrid)
Contract: Long Term
Required Skills & Experience
- Good years of overall Quality Engineering / Software Testing experience.
- Minimum 2+ years of hands-on experience in AI/ML testing, Generative AI evaluation, or LLM quality assessment.
- Strong programming skills in Python and experience developing and maintaining automated test solutions.
- Hands-on experience with API testing, integration testing, test automation, and regression testing.
- Practical knowledge of prompt testing, LLM evaluation, hallucination detection, and AI-generated response validation.
- Experience assessing AI applications for bias, safety, accuracy, reliability, and security vulnerabilities.
- Understanding of AI/ML concepts, model validation, and quality assurance practices for production-grade applications.
- Strong analytical, troubleshooting, documentation, and defect management skills.
- Ability to communicate testing results, risks, and recommendations to both technical and non-technical stakeholders.
More about this job
Responsibilities
Test and evaluate AI/ML applications and LLM-generated responses, including assessing prompts, hallucinations, bias, safety, accuracy, reliability, and security. Develop and maintain automated test solutions, perform API, integration, and regression testing, and communicate results, risks, and recommendations to stakeholders.
Requirements
Requires substantial quality engineering or software testing experience, including at least 2 years of hands-on AI/ML testing, Generative AI evaluation, or LLM quality assessment. Candidates should have strong Python and test automation skills, practical testing experience, knowledge of AI/ML and production quality practices, and strong analytical, documentation, troubleshooting, and communication abilities.
Skills
- Quality Engineering
- Software Testing
- AI/ML Testing
- Generative AI Evaluation
- LLM Quality Assessment
- Python
- Test Automation
- API Testing
- Integration Testing
- Regression Testing
- Prompt Testing
- Hallucination Detection
- AI Response Validation
- Bias and Safety Assessment
- Model Validation
- Defect Management
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Software
- Engineering
- Data & Analytics
Keywords
- Quality Engineering
- Software Testing
- AI/ML Testing
- Generative AI
- LLM Evaluation
- Python
- Automated Testing
- API Testing
- Integration Testing
- Regression Testing
- Prompt Testing
- Hallucination Detection
- AI-Generated Response Validation
- Bias Assessment
- Safety Assessment
- Accuracy
- Reliability
- Security Vulnerabilities
- Model Validation
- Quality Assurance
- Production-Grade Applications
- Troubleshooting
- Documentation
- Defect Management
- Stakeholder Communication