Find work in Canada.

524,639 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 1 min ago
M
Summer Intern 2027 - Private Wealth
Manulife
Montreal, QC
- Full-time
Opportunities to interact with senior leaders and gain insight into business decision-making within a leading financial services organization.
Added 15 mins ago
NB
Product Owner Regulatory Capital Transformation
National Bank of Canada
Montreal, QC
- Temporary
…Bank means acting as a change leader. You will report to the Senior Manager Business Intelligence Center - Finance. This role allows you to make a positive and tangible impact on the…
Added 15 mins ago
NB
Chief Functional Analyst
National Bank of Canada
Montreal, QC
- Full-time
You’ll be on a large team of 15 colleagues and report to the Senior Manager.
Added 15 mins ago
NB
Senior Account Manager, Real Estate Financing
National Bank of Canada
Montreal, QC
- Full-time
A career as a Senior Account Manager in the Real Estate Finance team of the Commercial & Private Banking 1859 sector at the National Bank means supporting the best players in the real…
Added 15 mins ago
CO
STRATEGIC ENTERPRISE INTEGRATION & MODERNIZATION LEAD (DATA MANAGEMENT)
City of Toronto
Toronto, ON
- Full-time
In this temporary role, reporting to the Senior Project Manager , the SEIM Lead (Data Management) will lead complex, multi-year, multi-divisional technology transformation projects for the…
Added 15 mins ago
NB
Senior Advisor Cybersecurity
National Bank of Canada
Montreal, QC
- Temporary
Required skills: Amazon Web Services, Cloud security, Communication, Cybersecurity standards, Mobilization, Negotiation, Software Development Life Cycle, Standards Compliance, Teamwork,…
Added 15 mins ago
NB
Analyst
National Bank of Canada
Montreal, QC
- Full-time
…expertise and sound risk management. This position reports to the Senior Manager of Capital Market…
Added 15 mins ago
NB
Business Development Manager
National Bank of Canada
Montreal, QC
- Full-time
A career as a Business Development Manager in the Distribution team at National Bank Insurance means acting as a strategic partner to help our distribution channels grow and better support…
Added 15 mins ago
NB
Senior Advisor Regulatory Governance
National Bank of Canada
Montreal, QC
- Full-time
Prepare executive dashboards, reports, and insights for senior management and governance committees while contributing to regulatory horizon scanning, training initiatives, and best…
Added 46 mins ago
AH
Senior Advisor
Alberta Health Services
AB · Greater Edmonton Metropolitan Area
- Full-time
…supported, and empowered to do their best work. We're looking for a Senior Advisor, Psychological Health & Safety (PHS) to join our People Experience team. This is a unique opportunity to…
Added 46 mins ago
AH
Operational Clinical Informatics Lead
Alberta Health Services
AB · Greater Edmonton Metropolitan Area
- Full-time
Transition Company: Health Shared Services Classification: Senior Project Manager Union: Exempt Unit and Program: Clinical Operations Informatics Office Primary Location: Aberhart Centre…
Added 46 mins ago
AH
Nuclear Medicine Technologist II
Alberta Health Services
Calgary, AB · Greater Calgary Metropolitan Area
- Temporary
…responsible for the day to day operations of Diagnostic Imaging services, providing operational leadership and supervision for multiple imaging modalities, including MRI and Nuclear…
Added 1 hour ago
M
RQ00828 - Software Developer - Full Stack - Senior
Maarut
Toronto, ON
- Contract
Senior-level experience developing Java and Node.js services, REST APIs, FHIR integrations, event publishers and consumers, and production application logic.
Added 1 hour ago
M
RQ00831 - DevOPS/Cloud Engineer - Senior
Maarut
Toronto, ON
- Contract
Senior-level AWS cloud engineering and DevOps experience building, operating, and troubleshooting production serverless, containerized, networking, security, and shared platform resources…
Added 2 hours ago
SL
Senior Software Engineer, Operations – Core Java & iOS focus
Sun Life
Toronto, ON · Waterloo, ON · Sun Life Waterloo King
- Full-time
We are looking for a Senior Software Engineer with strong full-stack Java experience and hands-on iOS development skills. You will develop and support reliable applications across the web,…
Added 2 hours ago
Plant Controller
Amrize
Saint-Constant, QC · St-Constant, Queb
The Controller is the “Business Partner” to assist in the development and implementation of product line objectives and strategies and ensure there is an effective control in the financial…
Added 2 hours ago
Senior Account Manager-Metro Vancouver (Guardteck)
Kandor
Burnaby, BC
Guardteck Security is a trusted Canadian-owned and operated provider of security services, proudly operating under Kandor Management Corp., a facility services provider alongside Everclean…
Added 4 hours ago
NS
Buyer
Nova Scotia Power
Halifax, NS
- Full-time
We are looking for a Buyer to join the Procurement team on a full-time basis. Reporting to the Senior Buyer, you will be responsible for providing excellent service by understanding…
Added 4 hours ago
AU
Security Guard (Floater) - Shopping Mall
Allied Universal
Winnipeg, MB
- $17 CAD / hour
- Part-time
We are North America's leading security and facility services provider with approximately 300,000 service personnel. At Allied Universal(R), we pride ourselves on fostering a promote from…
Added 4 hours ago
N
Software Engineer 4 - Kubernetes AI Platform Engineering
Nutanix
Vancouver, BC
- $151,000 to $227,000 CAD / year
- Full-time
You will report to the Senior Engineering Manager, who adopts a collaborative leadership style that encourages team involvement in decision-making and values open discussions. The manager…
Added 4 hours ago
GT
Senior Associate, Audit – Relocate to Australia
Grant Thornton Australia
Calgary, AB
- Full-time
Join a leading professional services firm in one of the world's most liveable countries
Added 4 hours ago
RC
Recreation Aide
Retirement Concepts
Courtenay, BC
- $29.62 to $31.4 CAD / hour
- Temporary
Assists in the development of recreation and leisure services including program research development, implementation and evaluation.
Added 4 hours ago
RC
Server
Retirement Concepts
Courtenay, BC
- $25.82 CAD / hour
- Full-time
…other with respect and dignity. We invest in the growth and development of our teams to actively participate in the creation of memories and moments that matter to our…
Added 4 hours ago
RC
Recreation Aide
Retirement Concepts
Courtenay, BC
- $27.92 CAD / hour
- Temporary
Assists in the development of recreation and leisure services including program research development, implementation and evaluation.
Added 4 hours ago
C
Maintenance & Asset Manager
Connective
Kamloops, BC
- Full-time
Connective is a dynamic non-profit organization that provides innovative programming in the social services sector. We strive to create safe, healthy, and inclusive communities for all.

Autodesk

Senior AI Quality Developer, Personalization

Toronto, ON, Canada

Full-time

Apply About employer

Applications open in a new tab.

Job details

Job type: Full-time
Apply by: Nov 28, 2026
Posting language: English
Posted: Oct 29, 2026

Original job description

Job Requisition ID #

26WD101776

Position Overview

At Autodesk, we're exploring new ways to make design and engineering software smarter, more intuitive, and more personal. AI is helping us rethink how millions of people interact with products like AutoCAD, Revit, Autodesk Construction Cloud, and Forma.

Our Personalized Experiences Team works across Autodesk to bring AI, machine learning, and personalization into our products. We collaborate with product teams, experiment with new ideas, and build capabilities that benefit customers across the Autodesk ecosystem.

We're looking for a Senior AI Quality Developer to help make those experiences reliable and trustworthy. In this role, you'll work on Autodesk Assistant, helping ensure we deliver high-quality, AI-powered experiences to our users. AI opens up exciting possibilities but also brings new challenges in how we test and evaluate software. You'll write code, build automation, solve interesting problems, and explore better ways to approach quality alongside software developers, applied scientists, and product teams.

More Info About Autodesk Assistant

https://www.autodesk.com/solutions/autodesk-ai/autodesk-assistant

Responsibilities

Design and implement quality engineering strategies for AI-powered personalization, recommendation systems, LLM applications, retrieval-augmented generation (RAG), and agentic systems
Build and maintain scalable automated functional, integration, API, end-to-end, and performance testing frameworks using TypeScript, Python, Playwright, Puppeteer, Pytest, or equivalent tools across web applications and cloud services
Develop golden datasets, benchmark suites, regression tests, and automated evaluation pipelines for nondeterministic AI behavior, including rubric-based, statistical, and LLM-as-a-judge evaluation methods
Evaluate RAG and personalization systems for retrieval quality, groundedness, relevance, ranking accuracy, consistency, and response quality across different user contexts
Test agentic workflows, including tool selection and execution, multistep orchestration, memory, error handling, authorization boundaries, and end-to-end task completion
Design automated quality checks for modular and distributed systems, including API contracts, integration boundaries, component compatibility, architectural regressions, and service reliability
Integrate automated tests and AI evaluations into CI/CD pipelines, establishing quality gates and release readiness criteria
Collaborate with applied scientists and developers to identify AI failure modes, evaluate security and privacy risks, and improve reliability, performance, observability, and production quality metrics
Champion quality engineering best practices, leverage AI-assisted development tools responsibly, and mentor developers across cross-functional teams

Minimum Qualifications

BS or MS in Computer Science, Software Engineering, or a related field, or equivalent practical experience
6 or more years of professional experience in software engineering, quality engineering, test automation, or related disciplines
Strong hands-on programming experience with TypeScript and Python, including designing and maintaining scalable automated testing frameworks
Experience with browser automation using Playwright, Puppeteer, or equivalent tools, and Python-based testing using Pytest or equivalent frameworks, covering functional, API, integration, and end-to-end testing
Practical experience testing or evaluating AI/ML-powered applications, including LLM-based systems, and developing automated evaluations for correctness, relevance, consistency, and reliability
Understanding of LLMs, embeddings, RAG, and AI agent architectures, including common failure modes, nondeterministic behavior, evaluation datasets, and regression testing approaches
Experience testing complex cloud services, distributed systems, or modular applications, including API contracts, integration boundaries, debugging, and system reliability
Experience with CI/CD pipelines, cloud environments (AWS preferred), performance testing, and production observability using logs, metrics, and traces
Understanding of data validation, security, privacy, and Responsible AI considerations when testing AI-powered systems
Ability to work independently, communicate technical findings clearly, and collaborate effectively with applied scientists, developers, and product partners

Preferred Qualifications

Experience with AI evaluation tools such as DeepEval, Ragas, LangSmith, promptfoo, Braintrust, or similar frameworks, including benchmark design, LLM-as-a-judge, and statistical or human evaluation methods
Experience testing agentic applications built using LangGraph, LangChain, or comparable orchestration frameworks, including adversarial testing and AI safety evaluation
Experience evaluating RAG systems using retrieval metrics, alongside groundedness, faithfulness, and answer quality
Experience with recommendation systems, ranking models, personalization, or search relevance, including offline and online experimentation and A/B testing
Familiarity with MLOps, model monitoring, dataset versioning, data validation pipelines, and production AI evaluation workflows
Experience evaluating AI inference performance, scalability, latency, throughput, and resource utilization in distributed or event-driven systems
Experience testing extensible or plugin-based applications, including API compatibility, SDKs, extension mechanisms, dependency validation, and consumer-driven contract testing
Experience using AI coding agents to develop reliable test automation, mentoring developers, or leading cross-team quality initiatives

Learn More / Plus d'information

About Autodesk / À Propos D’Autodesk

Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.

We take great pride in our culture here at Autodesk – it’s at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.

When you’re an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future? Join us!

Bienvenue chez Autodesk ! Nos logiciels créent chaque jour des choses extraordinaires : des bâtiments les plus écologiques aux voitures les plus propres, en passant par les usines les plus intelligentes et les films à succès. Nous aidons les innovateurs à concrétiser leurs idées, transformant non seulement la façon dont les choses sont fabriquées, mais aussi ce qui peut l’être.

Nous sommes très fiers de notre culture chez Autodesk ; elle est au cœur de tout ce que nous faisons. Notre culture guide notre façon de travailler et de nous comporter les uns envers les autres, influence nos interactions avec nos clients et nos partenaires, et définit notre image au monde.

En tant qu’Autodeskien, vous pouvez accomplir un travail significatif qui contribue à bâtir un monde meilleur, conçu et réalisé pour tous. Prêt à façonner le monde et votre avenir ? Rejoignez-nous !

Salary transparency / Transparence Salariale

Salary is one part of Autodesk’s competitive compensation package. For Canada based roles, we expect a starting base salary between $84,000 and $123,200. Offers are based on the candidate’s experience and geographic location, and may exceed this range. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package. Le salaire est l'un des éléments du programme de rémunération concurrentiel d'Autodesk. Pour les postes basés au Canada, nous offrons un salaire de base entre $84,000 et $123,200. Le salaire est déterminé selon l'expérience professionnelle et l'emplacement du candidat(e), et peut dépasser cette fourchette. En plus du salaire de base, notre programme de rémunération peut inclure une prime annuelle, des commissions pour les postes de ventes, des attributions d'actions et un ensemble complet d'avantages sociaux.

Belonging

We take pride in cultivating a culture of belonging where everyone can thrive. Learn more here: https://www.autodesk.com/company/global-belonging

In-Person Onboarding and Identity Verification

This role may require in-person onboarding and/or in-person ID verification.

Apply About employer

More about this job

Responsibilities

Design and build automated quality frameworks and evaluation pipelines for AI-powered personalization, LLM, RAG, recommendation, and agentic systems. Integrate testing and AI evaluations into CI/CD, assess reliability and safety, and collaborate with developers, applied scientists, and product teams to improve production quality.

Requirements

Requires a bachelor's or master's degree in computer science, software engineering, or a related field, or equivalent practical experience, plus at least six years of relevant professional experience. Candidates need strong TypeScript and Python skills, experience with automated testing and AI/ML applications, and knowledge of LLMs, RAG, cloud and distributed systems, CI/CD, performance, observability, security, and privacy.

Skills

TypeScript
Python
Playwright
Puppeteer
Pytest
AI Quality Engineering
Test Automation
Large Language Models
Retrieval-Augmented Generation
AI Evaluation
Distributed Systems
CI/CD
AWS
Performance Testing
Responsible AI
Recommendation Systems

Benefits

Annual Cash Bonuses
Commissions For Sales Roles
Stock Grants

Visa sponsorship

Not detected in the job text

Keywords

TypeScript
Python
Playwright
Puppeteer
Pytest
Large Language Models
Embeddings
Retrieval-Augmented Generation
AI Agents
Personalization
Recommendation Systems
Search Relevance
Automated Testing
Functional Testing
API Testing
Integration Testing
End-To-End Testing
Performance Testing
AI Evaluation
LLM-As-A-Judge
Benchmark Suites
Golden Datasets
CI/CD
AWS
Distributed Systems
MLOps
A/B Testing
Observability
Responsible AI
Security And Privacy

Report this job