QE Lead with AI Experience
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Experience
- 10+ years of experience in Quality Engineering, including 2+ years evaluating AI/ML systems.
- Posting language
- English
- Posted
Original job description
Job Title: QE Lead with AI Experience
Location: Toronto, ON
Work Arrangement: Hybrid (2-3 days a week)
Employment Type: Contract / Full Time
Salary/Pay Rate: Open to right candidate.
Domain: BFSI
Application Deadline: Nov. 13th, 2026
SKILLS REQUIRED:
This role is open for both full time and well as contract.
- 10+ years of experience in Quality Engineering, including 2+ years evaluating AI/ML systems.
- Strong expertise in Python, test automation, API testing, and integration testing.
- Experience designing prompt/response test sets and evaluating accuracy, groundedness, relevance, and hallucinations.
- Hands-on experience with regression, bias/safety, performance, penetration, and non-functional testing.
- Familiarity with GitHub Actions, GitHub Copilot, Claude Code, Copilot Studio, LangSmith, AppInsights, Log Analytics, and Dynatrace.
- Ability to provide auditable test evidence, defect analytics, and go/no-go recommendations for production readiness.
EEOC Compliance:
We are an equal opportunity employer, and all qualified applicants will receive consideration for employment.
DISCLAIMER
AI Usage Policy: Pacer Group uses AI to assist in screening applications. Final hiring decisions are made by human recruiters based on qualifications and experience.
More about this job
Responsibilities
Lead quality engineering and testing for AI/ML systems, including designing prompt-and-response test sets and evaluating accuracy, groundedness, relevance, and hallucinations. Provide auditable test evidence, defect analytics, and go/no-go recommendations for production readiness.
Requirements
Requires 10+ years of quality engineering experience, including at least 2 years evaluating AI/ML systems, with strong Python, test automation, API testing, and integration testing expertise. Candidates should have hands-on experience with regression, bias and safety, performance, penetration, and non-functional testing, and familiarity with the listed development and monitoring tools.
Skills
- Quality Engineering
- AI/ML Evaluation
- Python
- Test Automation
- API Testing
- Integration Testing
- Prompt and Response Test Design
- Accuracy Evaluation
- Groundedness Evaluation
- Relevance Evaluation
- Hallucination Testing
- Regression Testing
- Bias and Safety Testing
- Performance Testing
- Penetration Testing
- Non-Functional Testing
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Software
- Management & Leadership
- Finance & Accounting
Keywords
- Quality Engineering
- Artificial Intelligence
- Machine Learning
- Python
- Test Automation
- API Testing
- Integration Testing
- Prompt Testing
- Response Testing
- Accuracy
- Groundedness
- Relevance
- Hallucination Testing
- Regression Testing
- Bias Testing
- Safety Testing
- Performance Testing
- Penetration Testing
- Non-Functional Testing
- GitHub Actions
- GitHub Copilot
- Claude Code
- Copilot Studio
- LangSmith
- AppInsights
- Log Analytics
- Dynatrace
- Defect Analytics
- Production Readiness
- BFSI