Senior AI Scientist
- Montreal, QC, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Experience
- A bachelor’s degree in computer science, engineering, mathematics, statistics, data science, machine learning or a related field and 2 to 5 years of relevant experience
- Posting language
- English
- Posted
Original job description
A career as a Senior AI Scientist on the Model Risk Management team at National Bank means serving as an independent expert in the assessment of advanced artificial intelligence systems. This position allows you to have a concrete impact on the reliability, security and responsible use of the Bank's artificial intelligence solutions thanks to your expertise in AI assessment, quantitative analysis, software engineering and automation.
Your job
- Design methodologies, metrics, reference data sets and acceptance criteria for evaluating AI systems
- Assess LLMs, RAG systems, agents, voice, vision and multimodal solutions based on their risk profile
- Build, maintain, and evolve agentic solutions for the purposes of model validation
- Test adversarial tests, red teaming exercises and robustness, security and confidentiality analyses
- Develop test environments, automated pipelines and reusable tools to improve validation efficiency
- Clear, rigourous and evidence-based validation conclusions to inform decision-making
- Contribute to the evolution of standards, risk management practices and requirements related to the implementation of artificial intelligence solutions.
Within the Model Risk Management sector, you are part of a specialized growth team and report to the Senior Manager – AI Model Risk Management. Our team stands out for its combination of in-depth expertise in artificial intelligence, data and automation, as well as its rigour in risk management. She works closely with development teams while maintaining the independence needed to ensure effective validation of solutions.
Our goal is to offer you maximum flexibility to promote your quality of life. This includes a hybrid work environment with the possibility of working in Toronto, Montreal or Edmonton.
The Bank values continuous development and internal mobility. Our personalized training programs, based on-the-job learning, help you master your profession and develop new fields of expertise. Tools such as the Data Academy, language training, the Harvard Learning Center and coaching and mentoring support are available to you at any time.
Prerequisites
- A bachelor’s degree in computer science, engineering, mathematics, statistics, data science, machine learning or a related field and 2 to 5 years of relevant experience
- Experience in AI assessment, model validation, machine learning, software quality, red teaming or technology risk
- Experience designing, deploying or maintaining AI agents, LLM applications, RAG systems or AI-based automation
- Proficiency in Python, APIs, automated testing, data pipelines, version control and CI/CD practices
- Experience designing metrics, test data sets, evaluation frameworks or statistical experiments.
Reason to require this language: you will need to discuss frequently with our service providers or partners who speak a language other than French.
Required skills: Artificial Intelligence, Cybersecurity, Empathy, Initiative, Large Language Model, Learning Agility, Python, Resiliency, Risk Management, Teamwork
Your benefits
In addition to competitive compensation, upon hiring you’ll be eligible for a wide range of flexible benefits to help promote your wellbeing and that of your family such as:
* Health and wellness program, including many options
* Flexible group insurance
* Generous pension plan
* Employee Share Ownership Plan
* Employee and family assistance program
* Preferential banking services
* Involvement in community initiatives
* Telemedicine service
* Virtual sleep clinic
We have an offer that keeps up with trends as well as your needs and those of your family.
Our dynamic work environments and cutting-edge collaboration tools foster a positive employee experience. We value employees’ ideas. Whether through our surveys or programs, regular feedback and ongoing communication are encouraged.
Making a bold move in a people-first environment
We’re a bank on a human scale that stands out for its courage, entrepreneurial culture, and passion for people. Our mission is to have a positive impact on people’s lives. Our core values of partnership, agility, and empowerment inspire us, and inclusion is central to our commitments. We aim, wherever possible, to provide a barrier-free and accessible environment to all employees.
We strive to provide accessibility measures throughout the recruitment process within the limits of our available resources. If you require accommodations, feel free to let us know during our initial conversations. We welcome all candidates! What can you bring to our team?
Join us!
More about this job
Responsibilities
Design evaluation methodologies, metrics, datasets, and acceptance criteria, and independently assess AI systems such as LLMs, RAG systems, agents, and multimodal solutions. Build validation agents, test environments, automated pipelines, and reusable tools; conduct adversarial, robustness, security, and confidentiality testing; and provide evidence-based conclusions that inform risk decisions and standards.
Requirements
Requires a bachelor’s degree in computer science, engineering, mathematics, statistics, data science, machine learning, or a related field, plus 2–5 years of relevant experience. Candidates should have experience assessing or validating AI systems and designing or maintaining AI agents, LLM applications, RAG systems, or AI automation, along with proficiency in Python, APIs, automated testing, data pipelines, version control, CI/CD, and evaluation methods.
Skills
- Artificial Intelligence
- Cybersecurity
- Empathy
- Initiative
- Large Language Models
- Learning Agility
- Python
- Resilience
- Risk Management
- Teamwork
- Model Validation
- Machine Learning
- Red Teaming
- Software Engineering
- Automated Testing
- CI/CD
Benefits
- Health And Wellness Program
- Flexible Group Insurance
- Pension Plan
- Employee Share Ownership Plan
- Employee And Family Assistance Program
- Preferential Banking Services
- Community Initiatives
- Telemedicine Service
- Virtual Sleep Clinic
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Science & Research
- Finance & Accounting
- Security & Safety
Keywords
- Artificial Intelligence
- Model Risk Management
- Model Validation
- Large Language Models
- Retrieval-Augmented Generation
- AI Agents
- Multimodal AI
- Machine Learning
- Python
- APIs
- Automated Testing
- Data Pipelines
- Version Control
- CI/CD
- Evaluation Frameworks
- Metrics
- Test Data Sets
- Statistical Experiments
- Red Teaming
- Adversarial Testing
- Robustness Analysis
- Cybersecurity
- Confidentiality Analysis
- Software Engineering
- Automation
- Technology Risk
- Risk Management
- Voice AI
- Computer Vision