Find work in Canada.

328,151 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Save this search

Added 5 days ago
Automation Engineer 4- #26-23046
US Tech Solutions
Markham, ON
$65 to $70 USD / hour
Contract
We are looking for a dynamic, energetic experienced Systems Design Engineer to join our growing team. As a key contributor to the success of Client’s product, you will be part of a leading…
Added 1 month ago
AI/ML Engineer
TC Transcontinental
Mississauga, ON
Full-time
You care about good software design, notice the small details in code and API design, and take pride in building systems that are robust and scale well
Added 1 month ago
Project Engineer
Magna International
Brampton, ON
Full-time
Bachelor’s degree in Mechanical, Mechatronics, Software, or Systems Design Engineering
Added 2 months ago
Systems Engineer, Air & Naval Sonar, Intermediate/Working Level
General Dynamics Mission Systems–Canada
Ottawa, Ontario
Full-time
General Dynamics Mission Systems–Canada is looking for an experienced System Engineer to join our A&N Sonar team. In this role, you’ll lead systems engineering efforts that can span form…
Added 1 month ago
Life Cycle Material Engineering Specialist
General Dynamics Missions System International
Ottawa, ON
Full-time
At General Dynamics Mission Systems–Canada, we’re not just engineering technology - we’re shaping the future of defence and security. Our teams design and deliver advanced, mission-critical…
Added 11 days ago
Staff Software Engineer, AI/ML Infrastructure
Thumbtack
Remote, Canada
Full-time · Remote
As a Staff Software Engineer on the team, you will be a senior technical leader on a team whose systems sit in the critical path of every AI-powered experience at Thumbtack. You will set…
Added 2 months ago
Staff Software Development Engineer
Wagepoint
Remote, Canada
$140,000 to $170,000 CAD / year
Full-time · Remote
We're looking for a talented Staff Software Development Engineer who enjoys tackling hard problems and wants to create meaningful impact for small businesses across Canada. You will be a…
Added 2 months ago
Software Development Engineer, Amazon Connect Contact Lens
Amazon
Vancouver, British Columbia
- 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
Added 5 months ago
Agentic AI/ Edge Computing SW Engineer
Zebra Technologies
Mississauga, Ontario
Full-time
As the Agentic AI/ Edge Computing SW Engineer, you will help build the next generation of intelligent systems that bridge our hardware products with sophisticated backend AI processing. You…
Added 5 months ago
Staff Software Engineer - AI Platform
CaptivateIQ
Toronto, Ontario
Full-time
…manage sales performance. We're looking for a **Staff Software Engineer** to set the technical strategy for our Agentic SDK team, an internal developer platform that enables product teams…
Added 5 hours ago
Founding DevOps/SRE Engineer
Requesty
London, ON
Full-time
You understand that how to build security practices into systems in a way that protects the users but doesn't slow them down.
Added 16 hours ago
Cryptographic Systems Engineer (Post-Quantum)
SuperQubit Inc.
Markham, ON
Full-time
Job Title: Cryptographic Systems Engineer (Post-Quantum)
Added 17 hours ago
Senior Software Engineer- Telecom
Photon-IV
Canada
Full-time
The Senior Software Engineer – Telecom is a full-time, hybrid role based in Canada, with a mix of on-site collaboration and work-from-home flexibility. This role is responsible for leading…
Added 5 days ago
Manager, Software Engineering
AltaML
Toronto, ON
Full-time
…configuration and skills library are the default way engineers design, build, test, and review software, not an optional accelerant a few people…
Added 15 days ago
Applied Data Scientist, Health AI Evaluation & Datasets
Innodata Inc.
Canada
$210,000 to $240,000 USD / year
Full-time
Design evaluations for retrieval-augmented and source-grounded health AI systems, including evidence citation, faithfulness, contraindication handling, guideline adherence, source…
Added 4 days ago
Lead Software Engineer – Java/Spring, AWS & Solution Architecture
Element Fleet Management
Toronto, ON
Full-time
We’re seeking a hands-on Java/Spring technical leader who has led end-to-end solution design and delivery of API-driven, cloud-native applications on AWS. This is not a pure architecture or…
Added 2 days ago
Lead, Machine Learning Engineer, GFT
RBC
Toronto, ON
Full-time
Design and implement Generative AI solutions, including Retrieval-Augmented Generation (RAG) systems, LLM-powered agents, and NLP pipelines
Added 18 days ago
Software Engineer - Identity & Authorization
Baseten
Montreal, QC
$240,000 to $285,000 USD / year
Full-time
4-5+ years of experience building production backend systems, including hands-on design and implementation of a product authorization system: per-resource or per-object permissions,…
Added 4 days ago
Senior Application Developer
BMO
Calgary, AB · GCS
As a Senior Software Engineer at BMO, you will be a hands‑on technical leader responsible for the design, development, and ownership of complex, enterprise‑grade applications and services.…
Added 2 days ago
Senior Cloud DevOps Engineer
Charger Logistics Inc
Brampton, ON
Full-time
The Senior Cloud DevOps Engineer will design and implement enterprise-scale cloud infrastructure, drive automation strategies, and mentor engineering teams while architecting…
Added 2 days ago
Sr. Data Scientist
Charger Logistics Inc
Brampton, ON
Full-time
…ML, real-time and streaming analytics, and AI-driven decision systems built on cloud platforms, including Google Cloud, Kafka, and…
Added 2 days ago
Senior AI Engineer
Charger Logistics Inc
Brampton, ON
We are looking for a highly motivated AI Engineer to join our team based out of our Brampton office and contribute to the development of AI-driven solutions for various departments. This…
Added 1 day ago
Specialist Software Cloud Developer
CPKC
Calgary, AB
Full-time
Design, develop, and maintain ETL/ELT workflows to extract data from diverse sources including Oracle, SQL Server, SAP, SAP BW, Snowflake, AWS, Azure, GCP, and other enterprise systems
Added 1 day ago
Software Engineer
Actalent
Ottawa, ON
Contract
This Software Engineer role focuses on architecting, designing, and integrating advanced AI/ML capabilities into mission-critical defense software systems within a tactical IT environment.…
Added 2 days ago
Senior Software Engineer - Backend
Microsoft
Vancouver, BC
Full-time
Lead design and implementation of scalable, high-performance backend systems operating under strict latency and reliability requirements.

Innodata Inc.

Applied Data Scientist, Health AI Evaluation & Datasets

Canada

Pay: $210,000 to $240,000 USD / year
Job type: Full-time
Experience: 5+ years of data science experience, including at least 2+ years with healthcare, clinical, biomedical, payer, provider, pharma, life sciences, or comparable regulated health data.
Posting language: English
Posted: Sep 5, 2026

Apply About employer

Applications open in a new tab.

Original job description

Innodata (Nasdaq: INOD) is a global data engineering company. We believe that data and Artificial Intelligence (AI) are inextricably linked. Our mission is to enable the responsible advancement of artificial intelligence by providing the data, evaluation frameworks, and human expertise required to build AI systems that can be trusted at scale. We provide a range of transferable solutions, platforms, and services for Generative AI / AI builders and adopters. In every relationship, we honor our 36+ year legacy delivering the highest quality data and outstanding outcomes for our customers.

Scope Of The Role

Healthcare is one of the highest-stakes domains for generative AI. Clinical accuracy, patient safety, regulatory compliance, health equity, auditability, and workflow fit are the bar for shipping anything real. Innodata partners with foundation model labs, medical AI startups, payers, providers, pharma, and digital health companies building LLMs, multimodal systems, and AI agents for healthcare and life sciences.

As an Applied Data Scientist, Health AI Evaluation & Datasets, you own the design, measurement quality, and clinical validity of datasets used to train, fine-tune, and evaluate health-domain models. You bring clinical or biomedical fluency and data science rigor: you can read a clinical guideline, payer policy, medical literature artifact, or patient communication workflow; translate it into a measurable dataset and evaluation plan; and defend the methodology to sophisticated clinical, data science, and ML stakeholders.

You will work in a tight pod with a Technical Solutions Architect, Applied Research Scientist, AI/ML Research Engineer, and Language Data Scientists. Your role is to make sure the data, rubrics, review workflows, and measurement evidence are clinically realistic, statistically defensible, compliant, and useful for evaluation and post-training.

What You’ll Own

Translate customer goals - such as improving differential diagnosis, evaluating a clinical note summarizer, testing a RAG-based medical literature assistant, or creating preference data for patient-facing chatbots - into dataset specifications, taxonomies, rubrics, sampling plans, and acceptance criteria.
Make multimodal health AI a core focus: design training and evaluation datasets across clinical text, medical images, waveforms, structured EHR data, claims, trial data, medical literature, patient communications, payer policies, drug information, and other clinical artifacts, as well as use cases such as clinical reasoning, medical QA, note summarization, medical coding, patient communication, utilization management, and literature synthesis.
Design evaluations for retrieval-augmented and source-grounded health AI systems, including evidence citation, faithfulness, contraindication handling, guideline adherence, source freshness, and failure modes caused by incomplete, conflicting, or stale context.
Define sampling strategies, label schemas, inter-annotator agreement targets, adjudication workflows, SME review patterns, and quality thresholds in partnership with Language Data Scientists, clinicians, biomedical experts, and quality teams.
Build statistical and ML checks that make healthcare datasets trustworthy: stratified sampling across specialties and patient subgroups, bias and representation analysis, leakage detection, distribution shift checks, uncertainty estimates, reliability metrics, and subgroup performance analysis.
Partner with Applied Research Scientists and AI/ML Research Engineers to instrument datasets into evaluation and post-training pipelines, including rubric-grounded LLM-as-judge prompts, regression suites, model comparison workflows, experiment tracking, and model-improvement feedback loops.
Evaluate health AI behavior beyond surface accuracy: calibration, hallucination on safety-critical content, refusal appropriateness, robustness under ambiguity, equity across patient subgroups, and safe handoff in agentic or workflow-integrated systems. Reason concretely about clinical workflow fit: where outputs enter care delivery, what evidence a clinician or reviewer would need to trust them, when uncertainty must be surfaced, and how patient-facing, clinician-facing, payer, pharma, and operational use cases differ in risk.
Own data quality from source intake through delivery, including de-identified clinical text, medical literature, synthetic cases, structured records, client policies, and knowledge bases, with attention to PHI/PII handling, provenance, audit trails, versioning, and compliance documentation.
Stay current on the health AI landscape - regulatory developments such as FDA guidance on AI/ML-enabled medical devices and EU AI Act health provisions, benchmark releases such as MedQA, MedMCQA, and HealthBench, and emerging clinical evaluation methodology.
Support customer discovery and proposal work by scoping dataset programs, sizing annotation and SME review effort, identifying regulatory or data-access constraints, and explaining methodology choices to client clinical and ML leadership.
Contribute to Innodata internal IP: reusable health-domain taxonomies, evaluation rubrics, golden datasets, clinical review playbooks, dataset quality checks, and methodology templates.

You’ll Thrive In This Role If You Have

5+ years of data science experience, including at least 2+ years with healthcare, clinical, biomedical, payer, provider, pharma, life sciences, or comparable regulated health data.
Working knowledge of healthcare data and standards: EHR structure, clinical documentation conventions, ICD-10, CPT, SNOMED CT, LOINC, RxNorm, and at least passing familiarity with FHIR, HL7, or equivalent interoperability concepts.
Hands-on experience designing ML datasets, not just consuming them: writing annotation guidelines, sizing cohorts, setting quality thresholds, designing QA checks, and shipping data that downstream teams can train or evaluate on.
Familiarity with LLM-based health AI workflows, including prompt design, rubric-based evaluation, retrieval-augmented generation, LLM-as-judge methods, model comparison, and the limitations of automated evaluation in clinical contexts.
Strong Python and SQL; comfort with pandas, scikit-learn, statsmodels or equivalent tools; and working familiarity with modern LLM tooling such as Hugging Face, evaluation frameworks, prompt development tools, or model APIs.
Statistical literacy across sampling design, bias and fairness analysis, inter-annotator agreement metrics (Cohen or Fleiss kappa, Krippendorff alpha), confidence intervals, significance testing where appropriate, error analysis, and the ability to push back when a number is being over-interpreted.
Solid grasp of healthcare privacy, compliance, and governance: HIPAA, de-identification standards (Safe Harbor and Expert Determination), practical mechanics of working with PHI safely, auditability, access control, and documentation fit for high-stakes or regulated AI programs.
Ability to work credibly with clinicians, biomedical SMEs, research scientists, engineers, technical solutions teams, annotators, and customer stakeholders.
A bias toward clinical realism: you would rather build a smaller dataset that reflects what clinicians, reviewers, patients, or care teams actually see than a larger dataset that looks impressive on paper but fails in practice.
Degree in a relevant field such as biostatistics, epidemiology, computational biology, health informatics, computer science with a health focus, statistics, a clinical degree with quantitative training, or equivalent demonstrated experience.
Clinical credentials are not required, but candidates must be able to work credibly with clinicians, biomedical SMEs, and health AI customers; candidates with MD, RN, PharmD, MPH, PhD, or health informatics backgrounds are especially encouraged.

The expected salary range for this position is $210,000 – $240,000 USD per year, based on experience, skills, and qualifications.

Please be aware of recruitment scams involving individuals or organizations falsely claiming to represent employers. Innodata will never ask for payment, banking details, or sensitive personal information during the application process. To learn more on how to recognize job scams, please visit the Federal Trade Commission’s guide at https://consumer.ftc.gov/articles/job-scams.

If you believe you’ve been targeted by a recruitment scam, please report it to Innodata at [email protected] and consider reporting it to the FTC at ReportFraud.ftc.gov.

Apply About employer

More about this job

Responsibilities

Design and validate high-quality datasets and evaluation frameworks for health-domain AI models to ensure clinical accuracy and safety. Translate customer goals into technical specifications, rubrics, and sampling plans for multimodal health AI systems.

Requirements

Requires 5+ years of data science experience with at least 2 years in healthcare or life sciences and proficiency in Python and SQL. Candidates should have a strong grasp of healthcare data standards, privacy regulations like HIPAA, and LLM-based workflows.

Skills

Python
SQL
Pandas
Scikit-learn
LLM Evaluation
Retrieval-Augmented Generation
Statistical Sampling
Healthcare Data Standards
HIPAA Compliance
Dataset Design
Prompt Engineering
Bias Analysis
Clinical Reasoning
Medical Coding
Inter-annotator Agreement
Hugging Face

Visa sponsorship

Not detected in the job text

Keywords

Generative AI
LLM
Health AI
EHR
ICD-10
CPT
SNOMED CT
LOINC
RxNorm
FHIR
HL7
RAG
PHI
PII
MedQA
MedMCQA
HealthBench
FDA Guidance
EU AI Act
Biostatistics
Epidemiology
Computational Biology
Health Informatics
Statsmodels
Cohen's Kappa
Krippendorff's Alpha
Multimodal AI
Clinical Validity
Data Engineering
Model Calibration

Report this job