Software development jobs in Canada

Group jobs by employerEmail me new jobs Save this search

Added 22 days ago
IS
Cloud Engineer
Iris Software Inc.
Toronto, ON
- Contract
Added 22 days ago
B
Software Developer - Senior (User Interfaces, APIs, Services, Integrations - Lead exp)
Bevertec
Toronto, ON
- Contract
Must have experience leading teams and managing development teams
Added 22 days ago
IS
Sr. Devops Cloud Engineer (Terraform Development)
Iris Software Inc.
Toronto, ON
- Contract
Position: Sr. Devops Cloud Engineer (Terraform Development)
Added 22 days ago
Senior Software Engineer - Product Security
Faire
Toronto, ON · Waterloo, ON
Specific Workplace and Information Technology positions may require onsite attendance 5 days per week as will be indicated in the job posting
Added 22 days ago
SI
Java Backend Developer-Microservices, Spring, Angular
Smart IT Frame LLC
Mississauga, ON
- Full-time
…adherence to security and compliance standards in application development Optimize application performance and scalability based on monitoring and…
Added 22 days ago
TI
Full Stack Developer
Themesoft Inc.
Toronto, ON
- Contract
Executes on software development tasks or projects, requiring advanced problem solving, decision making and strategic thinking with some ambiguity.
Added 22 days ago
B
Intermediate to Senior Software Developer & AI Integrator - Montréal-based
Beslogic
Montreal, QC
- Full-time
At Beslogic, we build custom software and AI solutions for organizations with complex operations. Our clients work in industries like manufacturing, aerospace, logistics, and warehousing,…
Added 22 days ago
P
Full Stack Developer
PiiComm
ON · Plantagenet, ON
- Full-time
…applications in Azure with modern frameworks and AI-assisted development…
Added 22 days ago
UC
AI/Full Stack Developer
UniSysTech Consulting
Toronto, ON
- Other
…and be developed with engineering standards that make further development straightforward. The code should be fully covered by automated…
Added 23 days ago
CO
Staff Software Engineer - Capital One Travel Tech - Full-stack / développeur principal ou développeuse principale en logiciels
Capital One
Toronto, ON
- Full-time
Work as part of a dynamic team throughout the Software Development Life Cycle
Added 23 days ago
CC
Software Developer - Senior PeopleSoft HCM Developer (RQ11679)
Computer Consultants International, Inc.
Toronto, ON
- Contract
Candidates must be able to work onsite in Toronto five days per week
Added 23 days ago
Software Test Analyst
The Descartes
Waterloo, ON
Descartes is seeking an experienced Software Test Analyst to support quality engineering across the software development lifecycle. This automation-focused role is responsible for…
Added 23 days ago
CC
Software Developer - CRM Developer (RQ11698)
Computer Consultants International, Inc.
Toronto, ON
- Contract
Candidates must be able to work onsite in Toronto five days per week
Added 23 days ago
Software Engineer
Eastlink
Halifax, NS
Work Location: 6080 Young Street, Halifax, NS (On-site, 5 days per week)
Added 23 days ago
HT
Software Validation Technologist
Husky Technologies
Bolton, ON
- Full-time
The Software validation Technologist plays a critical role by working within the Innovation and Sustainability Team, helping to bring physical testing results to component and system…
Added 23 days ago
B
Senior Java Software Engineer
Bounteous
Montreal, QC
- Full-time
Experience with Git, BitBucket, and modern software development practices.
Added 23 days ago
W
Software Engineer (Python, Go)
Wiraa
Canada
- Full-time
As an Infrastructure Software Engineer at Baseten, you will play a critical role in developing and maintaining the core components of our machine learning inference platform. Your work will…
Added 23 days ago
N
Senior Software Development Engineer in Test (SDET), QA - AI Systems
Nexxa.AI
Remote, Canada
- Full-time
- Remote
5+ years as an SDET, QA engineer, or software engineer with a strong testing focus, with demonstrated ownership of test strategy and automation for complex distributed systems.
Added 23 days ago
J
Senior Microsoft Platform Software Engineer
Jobgether
Canada
- Full-time
Join an engineering team focused on modernizing software development practices for an established on-premises enterprise product.
Added 23 days ago
HT
Software Validation Technologist
Husky Technologies
Bolton, ON
- Full-time
The Software validation Technologist plays a critical role by working within the Innovation and Sustainability Team, helping to bring physical testing results to component and system…
Added 23 days ago
V
Full Stack Developer (Intermediate)
Vretta
Toronto, ON
- Full-time
…innovative assessment platforms. We are looking for an energetic Software Developer to join our fast-paced Product Development and Implementation team. In this role, you will take active…
Added 23 days ago
SC
Software Systems Engineer
SysEne Consulting Inc.
Vancouver, BC
- Full-time
You are an engineer with real depth in software, working across systems. You will bring software and data depth to our transit technology programs, where software runs real hardware and…
Added 23 days ago
SC
Software Systems Engineer
SysEne Consulting Inc.
Vancouver, BC
- Full-time
You are an engineer with real depth in software, working across systems. You will bring software and data depth to our transit technology programs, where software runs real hardware and…
Added 23 days ago
SM
RQ11679 – Software Developer – ERP / Case Management – Senior
S M Software Solutions Inc
Toronto, ON
- Internship
Work Arrangement: Onsite – 5 Days per Week
Added 23 days ago
SM
RQ11678 – Software Developer – ERP / Case Management – Senior
S M Software Solutions Inc
Toronto, ON
- Internship
Work Arrangement: Onsite – 5 Days per Week

Nexxa.AI

Senior Software Development Engineer in Test (SDET), QA - AI Systems

Remote, Canada

Remote

Full-time

Apply on jobs.ashbyhq.com About employer

Applications open in a new tab.

Job details

Job type: Full-time
Workplace: Remote
Remote eligibility: Canada listed. Check the posting for province restrictions.
Posting language: English
Posted: Oct 19, 2026

Original job description

About Nexxa

Nexxa is building the best AI systems for heavy industries, enabling machines, systems and operations to think, decide and act autonomously across manufacturing, large-scale infrastructure, logistics and legacy environments.

Our mission is to translate deep technical breakthroughs into operational reality, solving some of the hardest systems-level problems in industry.

Role Overview

We’re looking for a Senior / Staff SDET to own quality for Nexxa’s AI agent systems, combining deep test automation with hands-on manual and exploratory testing. Our agents plan, call tools, and take multi-step actions autonomously in industrial environments. You’ll write production-grade code to build evaluation harnesses, simulation environments, and CI/CD quality gates, and you’ll also get hands-on with the agents yourself: probing behavior, reviewing traces, and validating releases where automation alone can’t judge whether the agent did the right thing.

You’ll work closely with ML engineers, backend engineers, and Forward Deployed Engineers to define what “good” looks like for an agent operating in high-stakes industrial settings, then build and run the systems and processes that measure it continuously.

Key Responsibilities

Automation and Infrastructure

Design, build, and maintain test frameworks, evaluation harnesses, and regression suites for LLM-based agents, covering reasoning quality, tool-call correctness, task completion, and multi-turn coherence.
Build automated pipelines that run evals and test suites on every model, prompt, or tool-integration change, and integrate them into CI/CD as enforceable quality gates.
Develop simulators, mocks, and sandbox environments for industrial systems and tool APIs so agents can be tested safely, repeatably, and at scale.
Implement API, contract, integration, and end-to-end tests for the services, tools, and orchestration layers that agents depend on.
Develop performance, load, and reliability tests for agent workflows, including failure injection and resilience testing.

Manual and Exploratory Testing

Perform hands-on exploratory testing of agent behavior, including new capabilities, ambiguous or underspecified requests, long multi-turn sessions, and unusual operational scenarios that scripted tests don’t anticipate.
Manually review agent traces and outputs to judge reasoning quality, appropriateness of actions, and safety, and use those findings to improve automated checks.
Execute manual regression and release verification passes in staging and customer-like environments before new agent capabilities ship.
Conduct manual red-teaming sessions (prompt injection, jailbreaks, tool misuse, unsafe actions) in partnership with security teams, then convert successful attacks into automated test cases.
Support user acceptance testing and pilot rollouts with Forward Deployed Engineers, capturing real-world failures and feeding them back into test suites.
Write clear test plans, test charters, and bug reports, and maintain manual test documentation where automation isn’t yet practical or cost-effective.

Datasets, Metrics, and Quality Strategy

Create and manage golden datasets and labeled test sets, including edge cases, ambiguous inputs, and adversarial prompts specific to industrial and operational contexts, with tooling for versioning and drift detection.
Design rubrics and run human-in-the-loop evaluations where automated scoring isn’t reliable enough.
Define and instrument quality metrics beyond simple accuracy (groundedness, hallucination rate, task success rate, latency/cost tradeoffs, and safety violations) and build dashboards to track them over time.
Test agent behavior across the full action loop (planning, tool selection, tool execution, error recovery, and final output), not just the final response.
Investigate and triage failures where the root cause could be the model, the prompt, the tool/API, or the orchestration logic, and turn them into reproducible, actionable bug reports.
Establish quality bars, release criteria, and sign-off processes for new agent capabilities, blending automated gates with manual verification.
Decide what to automate and what to keep manual, and continually shift repeatable manual work into automation.
Mentor engineers on testing strategies and test-driven practices for probabilistic, LLM-driven systems.
Advocate for testability, observability, and tracing in agent architecture from day one.

Qualifications

5+ years as an SDET, QA engineer, or software engineer with a strong testing focus, with demonstrated ownership of test strategy and automation for complex distributed systems.
Proven experience in both manual and automated testing, including exploratory testing, test planning, and writing high-quality bug reports.
Strong programming skills (Python preferred) with a software engineering mindset: clean, maintainable, well-reviewed code for frameworks, data pipelines, and tooling.
Experience building test frameworks and infrastructure from the ground up, not just writing test cases within an existing framework.
Hands-on experience with CI/CD systems (e.g., GitHub Actions, GitLab CI, Jenkins) and containerized test environments (Docker, Kubernetes).
Experience testing APIs and microservices, including contract, integration, and end-to-end testing.
Hands-on experience testing LLM-based products, chatbots, or AI agents, and an understanding of why traditional deterministic assertions break down for generative systems.
Practical experience with eval frameworks or tooling (e.g., promptfoo, DeepEval, RAGAS, LangSmith) or a track record of building your own.
Understanding of how LLM agents work: prompting, tool/function calling, context management, RAG, memory, and orchestration frameworks.
Experience designing test data and labeled datasets, including sourcing, sampling, and managing dataset drift over time.
Familiarity with LLM-specific failure modes: hallucination, prompt injection, context poisoning, tool misuse, goal drift, and non-determinism.
Comfort operating in ambiguity, defining what “correct” means for a task when there’s no single right answer.
Strong written communication skills for turning fuzzy quality signals into clear, actionable findings for engineering and product stakeholders.

Preferred

Experience with human-in-the-loop evaluation workflows (labeling pipelines, inter-rater reliability, rubric design).
Experience with performance and load testing tools (e.g., k6, Locust) and chaos or fault-injection testing.
Background in ML/data science sufficient to read model evals and statistical significance.
Experience red-teaming or doing adversarial/security testing on ML systems.
Familiarity with observability and tracing tools for LLM applications (e.g., LangSmith, Arize, Langfuse, Weights & Biases) and general observability stacks (OpenTelemetry, Grafana, Datadog).
Experience testing systems in industrial, IoT, or operational technology (OT) environments, including industrial protocols (OPC UA, MQTT, Modbus) or simulation and hardware-in-the-loop setups.
Experience supporting UAT, pilots, or customer-facing validation of complex products.
Experience setting up test and eval infrastructure from scratch at a startup or fast-moving team.

What We’re Looking For

An engineer who is equally comfortable writing a test framework and sitting down to break an agent by hand.
Someone who can build rigorous evaluation systems for problems that don’t have a single right answer, and who knows when human judgment is the right tool.
A systems thinker who enjoys turning ambiguous agent behavior into measurable, trustworthy signals.
A strong collaborator who partners well with ML engineers, backend engineers, and Forward Deployed teams.

Why Join Nexxa.AI?

Innovative Environment: Play a critical role in transforming heavy industries through groundbreaking AI and automation technologies.
Collaborative Culture: Be part of a team that values innovation, discipline, and continuous improvement.
Professional Growth: Benefit from significant opportunities for career development and advancement.
Competitive Compensation: Enjoy a comprehensive salary and equity package reflective of your expertise and contributions.

If you’re passionate about AI quality and eager to help define what trustworthy autonomous systems look like in heavy industry, we’d love to connect.

Apply on jobs.ashbyhq.com About employer

More about this job

Responsibilities

You will design and maintain automated test frameworks and evaluation harnesses for LLM-based AI agents in industrial environments. Additionally, you will perform manual exploratory testing, red-teaming, and define quality metrics to ensure the reliability and safety of autonomous systems.

Requirements

The role requires 5+ years of experience in SDET or software engineering with a strong focus on testing complex distributed systems. Candidates must possess strong Python programming skills and practical experience with CI/CD pipelines, containerization, and testing generative AI or LLM-based products.

Skills

Python
Test automation
LLM evaluation
CI/CD
Exploratory testing
API testing
Distributed systems
Docker
Kubernetes
Prompt engineering
Performance testing
Data pipelines
Software engineering
Quality assurance
Debugging
System architecture

Benefits

Competitive salary
Equity package

Remote locations

Toronto, Canada

Remote locations (standardized)

Toronto, Ontario, Canada

Visa sponsorship

Not detected in the job text

Keywords

SDET
AI Systems
LLM
Python
Test Automation
CI/CD
GitHub Actions
Docker
Kubernetes
API Testing
Exploratory Testing
Red-teaming
Prompt Injection
RAG
LangSmith
DeepEval
Promptfoo
Distributed Systems
Quality Assurance
Industrial Automation
Observability
OpenTelemetry
Data Pipelines
Regression Testing
Performance Testing
Chaos Engineering
Software Engineering
Machine Learning
Evaluation Harnesses
Autonomous Systems

Report this job

Hiring at a glance

Advertised starting pay

Where work happens

Software development jobs in Canada

Cloud Engineer

Software Developer - Senior (User Interfaces, APIs, Services, Integrations - Lead exp)

Sr. Devops Cloud Engineer (Terraform Development)

Senior Software Engineer - Product Security

Java Backend Developer-Microservices, Spring, Angular

Full Stack Developer

Intermediate to Senior Software Developer & AI Integrator - Montréal-based

Full Stack Developer

AI/Full Stack Developer

Staff Software Engineer - Capital One Travel Tech - Full-stack / développeur principal ou développeuse principale en logiciels

Software Developer - Senior PeopleSoft HCM Developer (RQ11679)

Software Test Analyst

Software Developer - CRM Developer (RQ11698)

Software Engineer

Software Validation Technologist

Senior Java Software Engineer

Software Engineer (Python, Go)