Find work in Canada.

328,894 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 13 mins ago
AI
GCP Infrastructure / Cloud SRE Engineer
Apptoza Inc.
Toronto, ON
- Contract
Added 13 mins ago
FR
Condominium Manager
FirstService Residential Canada
Toronto, ON
- Full-time
Added 13 mins ago
AC
Property Field Claims Manager
Allstate Canada
ON · Ontario, Canada
- Full-time
Added 13 mins ago
E
Sr. Advisor Data Modernization
Enbridge
Calgary, AB
- Full-time
Added 13 mins ago
M
Commercial Transactions Analyst (Crude Oil / NGL)
Manpower
Calgary, AB
- Contract
Added 13 mins ago
LI
Quantity Surveyor - Calgary
LJB Inc.
Calgary, AB
- $150,000 to $170,000 CAD / year
- Full-time
Added 13 mins ago
MB
Structural steel Senior Estimator
Main Beam Fabrication Ltd.
Grande Prairie, AB
- Full-time
Added 13 mins ago
LM
Immunologist-Allergist
Lacroix Médecine Privée
Quebec, QC
- Full-time
Added 13 mins ago
AG
Réceptionniste de jour - Action Grenier Autogroupe - Daytime receptionist - Action Grenier Autogroupe
Action Grenier Autogroupe
Boucherville, QC
- Full-time
Added 13 mins ago
AM
Cost Estimator
Array Marketing
Toronto, ON · Greater Toronto Area, Canada
- $62,000 to $72,000 CAD / year
- Contract
Added 13 mins ago
CS
MongoDB Data Modeler / Data Engineer
Confiar Services
Toronto, ON
- Contract
Added 13 mins ago
SS
Manager, IT Business Systems
SMS Security
Toronto, ON
- $110,000 to $125,000 CAD / year
- Full-time
Added 13 mins ago
AM
Customs & Trade Compliance Analyst
Array Marketing
Toronto, ON · Greater Toronto Area, Canada
- Full-time
Added 13 mins ago
H
AI Solutions Architect
Hays
Toronto, ON
- Contract
Added 13 mins ago
LC
Service Representative - Waterloo Region
Libro Credit Union
Kitchener, ON
- Full-time
Added 13 mins ago
LM
Immunologist-Allergist
Lacroix Médecine Privée
Laval, QC
- Full-time
Added 13 mins ago
FC
Manager, CRC Energy Accounting
Federated Co-operatives Limited
Regina, SK
- Full-time
Added 13 mins ago
IG
Quality Engineer
Insight Global
Toronto, ON
- $40 to $45 CAD / hour
- Contract
Added 13 mins ago
FR
Condominium Manager
FirstService Residential Canada
Toronto, ON
- Full-time
Added 13 mins ago
BI
Insurance Broker
Billyard Insurance Group Inc.
Uxbridge, ON
- Full-time
Added 13 mins ago
SF
Director of Real Estate
Soul Foods Group
Toronto, ON · North York, ON
- $115,000 to $125,000 CAD / year
- Full-time
Added 13 mins ago
TS
Senior Property Manager, Commercial
TalentSphere Staffing Solutions
New Westminster, BC
- Full-time
Added 13 mins ago
EI
Firm Administrator
eStaffing Inc.
Langley (District), BC · Langley, BC
- Contract
Added 15 mins ago
R
Billingual Operations Representative
RBC
Calgary, AB · Vancouver, BC · Halifax, NS · Toronto, ON
- Full-time
Added 15 mins ago
R
Associate Advisor
RBC
Vancouver, BC
- Full-time

Iris Software Inc.

AI Tester

Toronto, ON, Canada
Greater Toronto Area, Canada, Canada

Contract

Apply About employer

Applications open in a new tab.

Job details

Job type: Contract
Apply by: Nov 6, 2026
Experience: 2 - 3 years of Experience in AI Tester.
Posting language: English
Posted: Oct 7, 2026

Original job description

We have Immediate requirement with our Direct Client as AI Tester @ Toronto ON (Hybrid - 4 Days/Week). if you are interested, Please share your updated resume to nakka.prasad@irissoftware.com

Title: AI Tester

Location: Toronto ON (Hybrid - 3 Days/Week)

Description:

2 - 3 years of Experience in AI Tester.

GenAI test specialist to bring deep expertise in testing LLM-powered features. You'll own how we evaluate AI quality, working within the existing test frameworks and CI/CD pipelines rather than building them from scratch.

What You'll DoDefine the GenAI testing strategy and quality metrics (accuracy, groundedness, hallucination, safety)
Agentic AI Framework, swarm testing, Agent orchestration – Claude/Devin
Build LLM evaluation suites, including LLM-as-a-judge, and plug them into our existing automation framework
Create and maintain golden datasets and adversarial prompt sets
Test RAG pipelines for retrieval quality, context relevance, and answer faithfulness
Validate AI agents and tool-calling flows across multi-step scenarios
Lead red-teaming: prompt injection, jailbreaks, bias, toxicity, PII leakage
Track model and prompt regressions across releases and share findings with ML and product teams
Upskill the wider QA team on GenAI testing practices
What You Bring3+ years in software testing or QA automation, with working Python skills
Hands-on experience testing LLM or GenAI applications
Strong grasp of LLM concepts: prompts, embeddings, context windows, temperature, RAG
Experience with eval tools such as DeepEval, Ragas, Promptfoo, LangSmith, or TruLens
Comfort with API testing and working inside an existing CI/CD setup
Nice to HaveLangChain / LlamaIndex and vector database experience
Knowledge of OWASP Top 10 for LLM Applications or AI red-teaming
ISTQB CT-AI or similar certification

Satya Prasad Nakka

Iris Software

200 Metroplex Drive, Suite #300

Edison, NJ 08817

Mobile: 732-426-9380

www.irissoftware.com

Apply About employer

More about this job

Responsibilities

Define GenAI testing strategies and quality metrics, build LLM evaluation suites, and maintain golden datasets and adversarial prompts within existing automation and CI/CD frameworks. Test RAG pipelines, AI agents, and tool-calling flows; lead red-teaming and regression tracking; and help upskill the QA team.

Requirements

The role requires 3 or more years in software testing or QA automation, working Python skills, and hands-on experience testing LLM or GenAI applications. Candidates should understand LLM concepts such as prompts, embeddings, context windows, temperature, and RAG, and have experience with evaluation tools, API testing, and existing CI/CD setups.

Skills

GenAI Testing
LLM Evaluation
Python
QA Automation
RAG Testing
API Testing
Prompt Engineering
Agentic AI
CI/CD
Red-Teaming
Test Strategy
Golden Datasets
Adversarial Testing
AI Safety
Regression Testing

Visa sponsorship

Not detected in the job text

Hiring contact

Satya Prasad Nakka

Hiring contact email

nakka.prasad@irissoftware.com

Keywords

GenAI
LLM
Python
DeepEval
Ragas
Promptfoo
LangSmith
TruLens
LangChain
LlamaIndex
Vector Databases
RAG
LLM-as-a-Judge
Agentic AI
Agent Orchestration
Claude
Devin
CI/CD
API Testing
Golden Datasets
Adversarial Prompts
Prompt Injection
Jailbreaking
AI Red-Teaming
OWASP Top 10 for LLM Applications
ISTQB CT-AI
Embeddings
Context Windows
Hallucination
Groundedness

Report this job