Slalom Responsible AI Practice · Toronto, ON · Full-time · Vacant Role
AI Risk Management & AI Risk Engineering - Consultant OR Sr. Consultant
Who You'll Work With
You'll join Slalom's Responsible AI practice and work on blended delivery teams with AI engineers, agentic AI and platform architects, MLOps and security engineers, and product owners, pairing on code and design rather than handing off documents. On the client side, you'll work with the engineering and platform teams who build and run AI systems, and with model risk, cybersecurity, privacy, legal, compliance and business owners who need to decide whether a system is ready for production and under what conditions.
Our work spans financial services, insurance, healthcare and life sciences, public sector, energy and utilities, and other regulated or high-consequence settings. Experience in any one of them is useful; experience in all of them is not expected.
What You'll Do
· Build and run evaluation suites for LLM, RAG and agentic systems, including scenario and benchmark packs, offline regression evaluations, tool-call and workflow tests and whole-trajectory evaluation, so release decisions rest on evidence rather than demos.
· Create and curate evaluation datasets (challenge and adversarial cases, human-labelled examples, cases derived from production incidents), and build automated graders, including LLM-as-judge, that you check against human judgement for validity, bias and drift.
· Design and execute automated testing of AI systems and agents (e.g., for direct and indirect prompt injection, data leakage, unsafe tool use, ungrounded or harmful outputs, and unfair outcomes); reproduce findings, rate their severity and turn them into tests and fixes.
· Analyze agent traces and failures to find root causes across planning, retrieval, memory, tool selection and hand-offs, then turn each confirmed failure into a regression test, a revised guardrail or an agent change.
· Implement runtime controls that make risk treatments enforceable: input and output guardrails, grounding checks, policy-as-code rules, action limits and budgets, human approval and escalation paths, and tested kill switches and rollback.
· Instrument AI systems for observability: capture traces, tool calls, interventions, quality, cost and latency signals, and set thresholds and alerts that detect behavioural drift and trigger targeted reassessment.
· Define and automate release gates in CI/CD and LLMOps pipelines, specifying the evidence a new agent or model version must produce and the results that block promotion, and support controlled pilots, versioned rollout and rollback.
· Contribute to AI platform and control-plane builds alongside architects, integrating evaluation, guardrail, policy, identity, registry and evidence components across AI gateways, MCP gateways and agent runtimes on AWS, Azure or Google Cloud.
· Assess AI use cases and agents for risk applying risk-tiering and impact-assessment methods, and translating the results into control requirements and evaluation plans proportionate to the stakes.
· Produce assurance evidence that holds up to scrutiny (evaluation results, control-test outcomes, system cards and traceable decision records) and map it to client policy and to frameworks such as the NIST AI RMF, ISO/IEC 42001, and regulatory frameworks.
· Explain technical findings to client engineers, risk partners and business owners so they can make informed go/no-go and autonomy decisions.
· Contribute reusable evaluation packs, control patterns, test harnesses and lessons learned to Slalom's Responsible AI practice, and provide technical input to client proposals when asked.
Core qualifications
· Professional experience building, deploying or testing ML, AI or software systems (3+ years for Consultant and 5+ years for Senior Consultant), including hands-on work on LLM-based applications such as RAG, tool-calling or agentic workflows, on at least one major cloud AI platform (e.g. Azure AI Foundry, AWS Bedrock, or GCP).
· Strong Python. You write maintainable, tested code for data handling, API integration, evaluation pipelines and automation, and you're comfortable working in someone else's codebase.
· Hands-on experience evaluating or testing AI systems. You have built evaluation workflows with repeatable datasets, metrics, automated graders or human-review protocols, and used the results to change a system or a release decision.
· Practical experience with at least one agent or LLM orchestration framework (for example, LangGraph, OpenAI Agents SDK, Strands Agents, Google ADK or Semantic Kernel), and a working understanding of how agentic systems fail: planning and tool-use errors, compounding error across steps, and memory and context issues.
· Experience implementing at least one kind of technical control or monitoring for an AI or software system, such as guardrails, authorization and IAM controls, policy enforcement, tracing and telemetry, alerting, or automated testing in CI/CD.
· Grounding in AI risk. You can identify credible failure modes and harms for a use case. Familiarity with at least one AI risk or model risk framework is expected.
· Clear written and verbal communication.
Preferred and differentiating experience
· Prior work in an AI evaluation, AI safety, red-team, Responsible AI engineering or AI assurance role, or published research or open-source contributions in AI evaluation or safety.
· Experience with evaluation and observability tooling such as DeepEval, RAGAS, LangSmith, Langfuse, Arize Phoenix, MLflow or OpenTelemetry.
· MCP server development or integration, AI or MCP gateway configuration, or agent identity and authorization patterns.
· Policy-as-code engines, or building policy enforcement into gateways and agent runtimes.
· MLOps or LLMOps experience, including CI/CD for models and agents, model and prompt registries, versioned rollout and production monitoring.
· Experimental design and statistics for evaluation: sampling, confidence intervals, inter-rater agreement and evaluator calibration.
· Experience with fairness assessment, explainability or human-AI interaction, including designing human oversight
· Experience in financial services, insurance, healthcare, public sector or another regulated setting, including work with model risk management, second-line risk or internal audit teams.
· Consulting or client-facing delivery experience. For Senior Consultant, experience owning a technical workstream and mentoring other engineers.
A degree in relevant field (e.g., computer science, policy, social sciences, engineering, or physical sciences) is required and advanced degree preferred. Experience working with or within highly regulated industries (e.g., financial services, healthcare, life sciences, public sector) is strongly preferred.
For the position of Consultant the expected base salary pay range is $120,000 to $145,000 and for Senior Consultant the expected base salary pay range is $145,000 to $170,000.
In addition, there is potential for a 8% discretionary bonus for the Consultant level and a 10% discretionary bonus for the Senior Consultant level and is based on individual and company performance. Actual compensation will depend upon individual’s skills, experience, qualifications, and other relevant factors.
Slalom Employee Perks:
- Culture & fun!
- Lunch & learns
- Health & wellness activities
- Holiday parties
- Employee community groups
- Social clubs & activities
- Vacation (20 days prorated based on start date & wellness/sick days)
Tangible perks:
- Competitive compensation package
- $100 cell phone monthly reimbursement
- Registered Retirement Program (RRSP) & Employer Matching
- Tax-Free Savings Account (TFSA)
- Annual “Thrive Well-Being” Gift Card
- Family planning/ Fertility Benefit
- Maternity/Parental Leave Top-Up
- Medical & Dental Benefits
- Life and AD&D Insurance
- Access to Supplementary Life and AD&D Insurance
- Access to the Employee Assistance Program (EAP) and Inkblot Therapy
- And more!
*All benefits are subject to eligibility requirements
Slalom is an inclusive, equal opportunity employer dedicated to building a diverse workforce. We encourage applications from all qualified candidates and will work to reasonably accommodate applicants’ needs throughout all stages of the recruitment and selection process. Please advise the talent acquisition team if you require accommodations during the interview process.
At Slalom, we leverage artificial intelligence (AI) to enhance the efficiency and scalability of our recruitment process. To learn more about how we leverage AI please read our TA AI Usage policy here.
Please note if you are hired at Slalom you will be required to complete a background check.
