Find work in Canada.

526,396 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 26 mins ago
YU
AI Agent Programming Assistant
York University
Toronto, ON
- Contract
The AI Agent Programming Assistant provides programming and application support for users in the academic departments within Lassonde School of Engineering, under the direction of the…
Added 42 mins ago
L
Senior BI Platform Operations Engineer
Luxoft
Toronto, ON
- Full-time
Added 42 mins ago
V
Python Developer
VySystems
Mississauga, ON
- Full-time
Hands-on Gen AI/LLM experience: prompt design, tool/function calling, and agent orchestration.
Added 56 mins ago
O
Senior Agentic AI Developer
OSEDEA
Montreal, QC
- Full-time
We're looking for a Senior Agentic AI Developer who can turn these capabilities into reliable, production-ready solutions for our clients.
Added 56 mins ago
O
Fullstack developer, agentic specialist
OSEDEA
Montreal, QC
- Full-time
We're looking for an Agentic AI Developer who enjoys building software, experimenting with emerging technologies, and turning ideas into working solutions.
Added 1 hour ago
GQ
Product Manager
Great Question
Remote, Canada
- Full-time
- Remote
…that chain simple and reliable enough that a builder, or an agent working for them, can run it end to end in a flexible way. That’s harder than it…
Added 2 hours ago
P
Senior Software Developer in Test | Ingénieur·e logiciel senior en test
Plotly
Remote, Canada
- Full-time
- Remote
We are seeking a highly skilled and experienced Senior Software Developer in Test (SDET) to join our dynamic team. As a Senior SDET, you will play a critical role in ensuring the quality…
Added 2 hours ago
HT
Intern Researcher- AI Agents
Huawei Technologies Canada Co., Ltd.
Markham, ON
- Internship
Develop and investigate new approaches for SFT, reinforcement learning, agent training, tool use, and function calling.
Added 2 hours ago
SF
Senior AI Engineer
Safe Fleet
Coquitlam, BC
- Full-time
We are looking for a hands-on Senior AI Engineer to build the platform that makes that possible - the shared services, agent framework, standards and tooling that allow every engineer on…
Added 2 hours ago
TL
Sales Development Representative
The Leadership Agency
Montreal, QC
- Full-time
Series A capital is going directly into product and GTM - including a sales ops agent for lead qualification and appointment setting
Added 2 hours ago
BI Analyst
PACT
ON · Mississauga, Ontario
…refreshes running without manual intervention, contributes to AI agent and Copilot work that lets stakeholders query program data directly, and works with business owners to translate…
Added 6 hours ago
AA
Principal Product Manager
Asensean Advisory
Remote, Canada
- $150,000 to $190,000 CAD / year
- Full-time
- Remote
Track customer needs, competitors, agent frameworks, model providers, and relevant policy and regulatory developments. Assess their implications across the product portfolio.
Added 7 hours ago
AI
GCP Architect
AllianceIT Inc
Canada
- Contract
Experience with LLM-based application architectures, ideally including agent-orchestrationframeworks such as LangGraph, LangChain and familiarity with the practical differences betweenLLM…
Added 10 hours ago
A
Forward Deployed Engineer
Antasphere
Brussels, ON
- Contract
Build that solution using our internal platform: products, templates, and agent tooling that generate most of the software for you.
Added 10 hours ago
L
Senior Product Designer, Developer Platform
LiveKit
Canada
- Full-time
…design for one very specific, very opinionated person: the developer at 2am trying to figure out why their voice agent just failed a call. You'll design the tools developers use to test,…
Added 11 hours ago
A
Senior AI Quality Developer, Personalization
Autodesk
Toronto, ON
- Full-time
We're looking for a Senior AI Quality Developer to help make those experiences reliable and trustworthy. In this role, you'll work on Autodesk Assistant, helping ensure we deliver…
Added 11 hours ago
A
Senior AI Quality Developer, Personalization
Autodesk
Toronto, ON
- Full-time
We're looking for a Senior AI Quality Developer to help make those experiences reliable and trustworthy. In this role, you'll work on Autodesk Assistant, helping ensure we deliver…
Added 12 hours ago
UW
Applied AI Engineer (Contract)
Upstream Works Software
Canada
- Contract
Our platform brings AI into the contact centre, and this engagement will help us prove agent quality before it reaches customers.
Added 13 hours ago
MP
Junior Real Estate Agent – Dubai (Training Provided)
McCone Properties
Ottawa, ON
- Full-time
The average agent here earned US$112,000 last year, tax free; US$2.9 million is our best year, not the usual one. There is no salary, so arrive with around six months of savings while you…
Added 14 hours ago
M
AI Business Solutions
Microsoft
Calgary, AB · Edmonton, AB · Vancouver, BC · Winnipeg, MB
- Full-time
…business requirements into prioritized Copilot scenarios and agent solutions, and drive deployment, active usage, adoption, and expansion across the customer's…
Added 16 hours ago
SC
Senior Developer
SAP Careers
Montreal, QC
- $108,100 to $222,800 CAD / year
- Full-time
We are looking for a dynamic and motivated Senior Software Developer who embrace AI-assisted software development as part of their everyday engineering practice to join our engineering…
Added 16 hours ago
F
React JS Developer
Fortinet
Burnaby, BC
- Full-time
…data everywhere. We are currently seeking a dynamic ReactJS Developer to contribute to the success of our rapidly growing business. We are looking for a highly motivated individual who can…
Added 17 hours ago
C
Senior Agentic AI developer
CGI
Toronto, ON
- Full-time
…(GTO) Canada is seeking a highly skilled Senior Agentic AI Developer to design, develop, and deploy next-generation AI solutions leveraging Large Language Models (LLMs), Generative AI,…
Added 18 hours ago
A
Entry Level Advisory Development Program - Summer Analyst - Canada - NAELFY27
Accenture
Calgary, AB · Vancouver, BC · Ottawa, ON · Toronto, ON · Montreal, QC
- Full-time
Added 18 hours ago
P
Experienced Talent Acquisition Manager
PwC
Calgary, AB · Toronto, ON
- Full-time

Autodesk

Senior AI Quality Developer, Personalization

Toronto, ON, Canada

Full-time

Apply About employer

Applications open in a new tab.

Job details

Job type: Full-time
Apply by: Nov 28, 2026
Posting language: English
Posted: Oct 29, 2026

Original job description

Job Requisition ID #

26WD101776

Position Overview

At Autodesk, we're exploring new ways to make design and engineering software smarter, more intuitive, and more personal. AI is helping us rethink how millions of people interact with products like AutoCAD, Revit, Autodesk Construction Cloud, and Forma.

Our Personalized Experiences Team works across Autodesk to bring AI, machine learning, and personalization into our products. We collaborate with product teams, experiment with new ideas, and build capabilities that benefit customers across the Autodesk ecosystem.

We're looking for a Senior AI Quality Developer to help make those experiences reliable and trustworthy. In this role, you'll work on Autodesk Assistant, helping ensure we deliver high-quality, AI-powered experiences to our users. AI opens up exciting possibilities but also brings new challenges in how we test and evaluate software. You'll write code, build automation, solve interesting problems, and explore better ways to approach quality alongside software developers, applied scientists, and product teams.

More Info About Autodesk Assistant

https://www.autodesk.com/solutions/autodesk-ai/autodesk-assistant

Responsibilities

Design and implement quality engineering strategies for AI-powered personalization, recommendation systems, LLM applications, retrieval-augmented generation (RAG), and agentic systems
Build and maintain scalable automated functional, integration, API, end-to-end, and performance testing frameworks using TypeScript, Python, Playwright, Puppeteer, Pytest, or equivalent tools across web applications and cloud services
Develop golden datasets, benchmark suites, regression tests, and automated evaluation pipelines for nondeterministic AI behavior, including rubric-based, statistical, and LLM-as-a-judge evaluation methods
Evaluate RAG and personalization systems for retrieval quality, groundedness, relevance, ranking accuracy, consistency, and response quality across different user contexts
Test agentic workflows, including tool selection and execution, multistep orchestration, memory, error handling, authorization boundaries, and end-to-end task completion
Design automated quality checks for modular and distributed systems, including API contracts, integration boundaries, component compatibility, architectural regressions, and service reliability
Integrate automated tests and AI evaluations into CI/CD pipelines, establishing quality gates and release readiness criteria
Collaborate with applied scientists and developers to identify AI failure modes, evaluate security and privacy risks, and improve reliability, performance, observability, and production quality metrics
Champion quality engineering best practices, leverage AI-assisted development tools responsibly, and mentor developers across cross-functional teams

Minimum Qualifications

BS or MS in Computer Science, Software Engineering, or a related field, or equivalent practical experience
6 or more years of professional experience in software engineering, quality engineering, test automation, or related disciplines
Strong hands-on programming experience with TypeScript and Python, including designing and maintaining scalable automated testing frameworks
Experience with browser automation using Playwright, Puppeteer, or equivalent tools, and Python-based testing using Pytest or equivalent frameworks, covering functional, API, integration, and end-to-end testing
Practical experience testing or evaluating AI/ML-powered applications, including LLM-based systems, and developing automated evaluations for correctness, relevance, consistency, and reliability
Understanding of LLMs, embeddings, RAG, and AI agent architectures, including common failure modes, nondeterministic behavior, evaluation datasets, and regression testing approaches
Experience testing complex cloud services, distributed systems, or modular applications, including API contracts, integration boundaries, debugging, and system reliability
Experience with CI/CD pipelines, cloud environments (AWS preferred), performance testing, and production observability using logs, metrics, and traces
Understanding of data validation, security, privacy, and Responsible AI considerations when testing AI-powered systems
Ability to work independently, communicate technical findings clearly, and collaborate effectively with applied scientists, developers, and product partners

Preferred Qualifications

Experience with AI evaluation tools such as DeepEval, Ragas, LangSmith, promptfoo, Braintrust, or similar frameworks, including benchmark design, LLM-as-a-judge, and statistical or human evaluation methods
Experience testing agentic applications built using LangGraph, LangChain, or comparable orchestration frameworks, including adversarial testing and AI safety evaluation
Experience evaluating RAG systems using retrieval metrics, alongside groundedness, faithfulness, and answer quality
Experience with recommendation systems, ranking models, personalization, or search relevance, including offline and online experimentation and A/B testing
Familiarity with MLOps, model monitoring, dataset versioning, data validation pipelines, and production AI evaluation workflows
Experience evaluating AI inference performance, scalability, latency, throughput, and resource utilization in distributed or event-driven systems
Experience testing extensible or plugin-based applications, including API compatibility, SDKs, extension mechanisms, dependency validation, and consumer-driven contract testing
Experience using AI coding agents to develop reliable test automation, mentoring developers, or leading cross-team quality initiatives

Learn More / Plus d'information

About Autodesk / À Propos D’Autodesk

Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.

We take great pride in our culture here at Autodesk – it’s at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.

When you’re an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future? Join us!

Bienvenue chez Autodesk ! Nos logiciels créent chaque jour des choses extraordinaires : des bâtiments les plus écologiques aux voitures les plus propres, en passant par les usines les plus intelligentes et les films à succès. Nous aidons les innovateurs à concrétiser leurs idées, transformant non seulement la façon dont les choses sont fabriquées, mais aussi ce qui peut l’être.

Nous sommes très fiers de notre culture chez Autodesk ; elle est au cœur de tout ce que nous faisons. Notre culture guide notre façon de travailler et de nous comporter les uns envers les autres, influence nos interactions avec nos clients et nos partenaires, et définit notre image au monde.

En tant qu’Autodeskien, vous pouvez accomplir un travail significatif qui contribue à bâtir un monde meilleur, conçu et réalisé pour tous. Prêt à façonner le monde et votre avenir ? Rejoignez-nous !

Salary transparency / Transparence Salariale

Salary is one part of Autodesk’s competitive compensation package. For Canada based roles, we expect a starting base salary between $84,000 and $123,200. Offers are based on the candidate’s experience and geographic location, and may exceed this range. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package. Le salaire est l'un des éléments du programme de rémunération concurrentiel d'Autodesk. Pour les postes basés au Canada, nous offrons un salaire de base entre $84,000 et $123,200. Le salaire est déterminé selon l'expérience professionnelle et l'emplacement du candidat(e), et peut dépasser cette fourchette. En plus du salaire de base, notre programme de rémunération peut inclure une prime annuelle, des commissions pour les postes de ventes, des attributions d'actions et un ensemble complet d'avantages sociaux.

Belonging

We take pride in cultivating a culture of belonging where everyone can thrive. Learn more here: https://www.autodesk.com/company/global-belonging

In-Person Onboarding and Identity Verification

This role may require in-person onboarding and/or in-person ID verification.

Apply About employer

More about this job

Responsibilities

Design and build scalable automated testing and evaluation systems for AI-powered personalization, recommendation, RAG, LLM, and agentic applications. Integrate quality checks into CI/CD, assess reliability and safety, and collaborate with scientists, developers, and product teams to improve production quality.

Requirements

Requires a bachelor's or master's degree in computer science, software engineering, or a related field, or equivalent practical experience, plus at least six years of relevant professional experience. Candidates need strong TypeScript and Python skills, experience with automated testing and AI/ML application evaluation, and familiarity with cloud services, distributed systems, CI/CD, security, privacy, and Responsible AI.

Skills

TypeScript
Python
AI Quality Engineering
Test Automation
Playwright
Puppeteer
Pytest
Large Language Models
Retrieval-Augmented Generation
AI Evaluation
Recommendation Systems
CI/CD
Distributed Systems
AWS
Performance Testing
Responsible AI

Benefits

Annual Cash Bonuses
Stock Grants

Visa sponsorship

Not detected in the job text

Keywords

TypeScript
Python
Playwright
Puppeteer
Pytest
Large Language Models
Retrieval-Augmented Generation
Embeddings
AI Agents
Personalization
Recommendation Systems
Search Relevance
AI Evaluation
LLM-as-a-Judge
Golden Datasets
Benchmark Suites
Regression Testing
API Testing
End-to-End Testing
Performance Testing
CI/CD
AWS
Distributed Systems
MLOps
A/B Testing
LangGraph
LangChain
DeepEval
Ragas
Responsible AI

Report this job