Find work in Canada.

524,639 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Email me new jobs Save this search

Added 1 hour ago
Technical Solutions Development Manager, Workspace, Google Cloud
Google
Waterloo, ON
You will lead complex, multi-disciplinary projects from inception to completion, defining requirements, managing schedules, and mitigating risks. In this critical role, you will help…
Added 6 hours ago
UO
Senior Web Developer
University of Ottawa
Ottawa, ON · Tabaret Hall
- Full-time
Provides strategic and operational leadership in the planning, development, and implementation of the University’s Web sites. Ensures the integrated management (leadership, planning, and…
Added 6 hours ago
UW
Applied AI Engineer (Contract)
Upstream Works Software
Canada
- Contract
Upstream Works is a Canadian software company with 25 years of experience in enterprise contact centre software, trusted by Cisco, Amazon Web Services and global partners.
Added 6 hours ago
BK
Manager, Graphic Design, Burger King, Canada
Burger King
Toronto, ON
- Full-time
Strong knowledge of web design and development tools, including HTML, CSS, JavaScript, and content management systems (CMS).
Added 10 hours ago
A
Software Development Manager, Books Discovery
Amazon
Toronto, ON
- Full-time
As a Software Development Manager, you will lead a collection of skilled, friendly people from all over the world and collaborate with business owners to build a clear roadmap of priorities…
Added 12 hours ago
A
Entry Level Technology Development Program - Summer Analyst - Canada - NAELFY27
Accenture
Calgary, AB · Vancouver, BC · Ottawa, ON · Toronto, ON · Montreal, QC · Regina, SK
- Full-time
Identify, prioritize, and document high-impact opportunities for AI integration across client projects (e.g., customer support, research, legal review, software development).
Added 12 hours ago
O
Sr. Software Developer
OpenText
Richmond Hill, ON
- Full-time
Serve as a technical lead, recognized for growing domain expertise, embracing change, and navigating ambiguity with resiliency, while fostering the development of less experienced team…
Added 13 hours ago
VP Sales & Marketing (Bilingual Eng/Fre)
Banyan Software
Montreal, QC
Banyan Software is the best permanent home for software businesses that serve specialized industries, their employees, and their customers. With a buy-grow-and-hold-for-life approach and a…
Added 14 hours ago
RB
Manager, Graphic Design, Burger King, Canada
Restaurant Brands International
Toronto, ON · Corp - Toronto Corporate Office
- Full-time
Strong knowledge of web design and development tools, including HTML, CSS, JavaScript, and content management systems (CMS).
Added 14 hours ago
RI
Senior Full Stack Engineer
Remarcable, Inc.
Remote, Canada
- $140,000 to $190,000 CAD / year
- Full-time
- Remote
We are expanding our Vancouver engineering team - our core product and development hub - and looking for a Senior Full Stack Engineer to take technical ownership of high-impact product…
Added 14 hours ago
E
Braze Architect
EXL
Canada
- Full-time
…(Hightouch, Amperity, Amplitude, Snowflake, and similar). You will lead enterprise-scale implementations and migrations, set engineering and governance standards, advise senior client…
Added 15 hours ago
TF
Software Development Lead
Turner Fleischer
Toronto, ON
- Full-time
As Software Development Lead, you oversee the design, development, and maintenance of custom software applications that support and streamline the studio’s business operations. Leading a…
Added 15 hours ago
T
Software Engineer II (SDET)
TD
Toronto, ON
- Full-time
Participate in personal performance management and development activities, including cross training within own team
Added 15 hours ago
SH
Software Developer - Enhance Ontario
Scarborough Health Network
EO Head Office
- Full-time
As a senior member of the Software and Development Team, the Senior Software Developer takes the lead on designing, architecting, and delivering web applications and integration solutions…
Added 15 hours ago
DL
Salesforce Omnistudio Developer
Diverse Lynx
ON · Ontario, Canada
- Contract
Lead technical design, development, code reviews, and end-to-end solution delivery.
Added 16 hours ago
G
Chef(fe) d'équipe tests de production / Test Software Team Lead - Manufacturing
Genetec
Montreal, QC
- Full-time
The job involves collaborating with R&D (Electronics, Firmware, Software, Mechanics), suppliers, and manufacturers to bring new products into production while ensuring the quality of…
Added 16 hours ago
G
Chef(fe) d'équipe tests de production / Test Software Team Lead - Manufacturing
Genetec
Remote, Canada
- Full-time
- Remote
The job involves collaborating with R&D (Electronics, Firmware, Software, Mechanics), suppliers, and manufacturers to bring new products into production while ensuring the quality of…
Added 16 hours ago
Account Development Representative
MongoDB
Toronto, ON
At MongoDB, our Account Development team works closely with our partners in both Sales and Marketing to build fanatical customer enthusiasm around MongoDB. ADR reps are responsible for…
Added 16 hours ago
Senior Security Engineer
Dye & Durham Talent Team
Toronto, ON
Dye & Durham, a leading global provider of cloud–based software and technology solutions, provides critical information services and workflows used by clients all over the world to manage…
Added 16 hours ago
AF
Lead UX Designer
Armstrong Fluid Technology
Toronto, ON
- Full-time
Lead and mentor the UX design team, review its work, and build a strong culture of user-centered design across the development team.
Added 17 hours ago
CO
Temporary Solutions Integration Specialist (up to 8 months)
City of Saint John
Saint John, NB
- Temporary
…administration, implementation, and maintenance of the City’s website, web-based applications, software, data (including databases and interfaces), enterprise system and other…
Added 17 hours ago
CO
Application & Integration Technical Analyst
City of Saint John
Saint John, NB
- Full-time
Contribute to the development and implementation of relevant IT policies, standards, and operating procedures within the team’s technical domain
Added 17 hours ago
TC
M365 Technical Lead
Tata Consultancy Services
Vancouver, BC
- Full-time
Lead development of SPFx web parts/extensions, reusable components and integrations using TypeScript, React, JavaScript, REST APIs, Microsoft Graph and approved development standards.
Added 17 hours ago
M
Senior Consultant, Advanced Analytics
MNP
Edmonton, AB · AB · NS · Calgary, AB · Dartmouth, NS
- Full-time
…collaborate with our Business Intelligence, Data & Analytics team to design and deliver modern cloud data platforms that enable business intelligence, artificial intelligence, advanced…
Added 17 hours ago
M
Senior Consultant, Advanced Analytics
MNP
Vancouver, BC · Ottawa, ON · Toronto, ON
- Full-time
…collaborate with our Business Intelligence, Data & Analytics team to design and deliver modern cloud data platforms that enable business intelligence, artificial intelligence, advanced…

Upstream Works Software

Applied AI Engineer (Contract)

Canada

Contract

Apply About employer

Applications open in a new tab.

Job details

Job type: Contract
Posting language: English
Posted: Oct 27, 2026

Original job description

About The Role

Upstream Works Software is hiring a contract Applied AI Engineer to contribute to the setup of our agentic AI evaluation and demo environment on Amazon Bedrock AgentCore.

Upstream Works is a Canadian software company with 25 years of experience in enterprise contact centre software, trusted by Cisco, Amazon Web Services and global partners.

Our platform brings AI into the contact centre, and this engagement will help us prove agent quality before it reaches customers.

You will own the hands on build: setting up AgentCore, creating a golden dataset and evaluation process, and delivering an end to end demo shaped around a real business use case. You will work closely with our Product Lead, who will define the business requirements the demo needs to meet.

Key Responsibilities

The work runs in three phases, each with a clear deliverable.

Setup AWS AgentCore

Lead the setup of Amazon Bedrock AgentCore in our AWS environment, including Runtime, Observability and Evaluations.
Configure secure access, permissions and logging following AWS best practices.
Document the setup so our team can maintain and extend it.

Build the golden dataset and evaluation process

Work with our Product Lead and subject matter experts to define realistic scenarios, expected responses and the steps the agent should take.
Build and version a golden dataset that reflects real contact centre interactions, using anonymized or synthetic data.
Design the evaluation approach, including success criteria, built in and custom evaluators, and a repeatable process for regression testing as the agent changes.
Report on agent quality in a way business stakeholders can understand.

Build an end to end demo in AgentCore

Build a working agent and connect it to the tools and data the use case needs.
Shape the demo around the business requirements defined with our Product Lead, and iterate based on feedback.
Prepare and present the demo to internal stakeholders.

Throughout the engagement, you will share regular progress updates, flag risks early, and leave behind clear documentation and reusable components.

Required Qualifications

3 or more years of hands on software or AI engineering experience, including building at least one LLM based application or agent that reached production or real users
Hands on experience with Amazon Bedrock and Amazon Bedrock AgentCore
Strong Python skills, including APIs, SDKs and data handling
Experience building AI agents with tool or function calling, retrieval, and at least one agent framework (for example, Strands, LangGraph, CrewAI or similar)
Experience designing evaluations for non deterministic AI systems, such as golden datasets, success criteria, LLM as a judge, and regression testing
Working knowledge of core AWS services, including IAM, CloudWatch, Lambda and S3
Ability to turn business requirements into a working technical solution with limited direction
Excellent written and verbal English, and the ability to explain technical results to non-technical stakeholders
Availability for regular meetings during Eastern Time business hours

Would Be An Asset

Experience with conversational AI or voice agents (for example, Amazon Connect, Cisco or Genesys)
AWS certifications, such as AWS Certified AI Practitioner, Machine Learning Engineer Associate or Solutions Architect Associate
Experience with Model Context Protocol (MCP) or AgentCore Gateway
Experience setting up evaluation in CI/CD pipelines
Experience handling sensitive customer data, including anonymization and privacy requirements

Apply About employer

More about this job

Responsibilities

Set up and document Amazon Bedrock AgentCore, including secure access, observability, and evaluation capabilities. Develop a versioned golden dataset and repeatable evaluation process, then build, refine, and present an end-to-end agent demo based on business requirements.

Requirements

Requires at least three years of hands-on software or AI engineering experience, including a production or real-user LLM application, plus experience with Amazon Bedrock AgentCore, Python, AWS, agent frameworks, and AI evaluation. Candidates must be able to translate business needs into technical solutions, communicate results clearly in English, and attend regular meetings during Eastern Time business hours.

Skills

Amazon Bedrock AgentCore
Python
AWS
AI Agent Development
Tool And Function Calling
Retrieval-Augmented Generation
AI Evaluation
Golden Dataset Development
Regression Testing
IAM
CloudWatch
Lambda
S3
API And SDK Integration
Data Handling
Stakeholder Communication

Visa sponsorship

Not detected in the job text

Keywords

Amazon Bedrock
Amazon Bedrock AgentCore
AWS
Python
LLM
AI Agents
Strands
LangGraph
CrewAI
Tool Calling
Function Calling
Retrieval
Golden Dataset
AI Evaluation
LLM As A Judge
Regression Testing
IAM
CloudWatch
AWS Lambda
Amazon S3
Amazon Connect
Cisco
Genesys
Model Context Protocol
AgentCore Gateway
CI/CD
Conversational AI
Voice Agents
Data Anonymization
Contact Centre Software

Report this job