Software Developer - Senior

Project overview

The Government of Ontario is undertaking a major Data and AI transformation initiative focused on developing the Ontario Data Platform (ODP).

The ODP is an enterprise technology ecosystem designed to support data and AI solutions across government ministries, clusters, and program areas.

The platform includes:

• Data ingestion technologies.

• Data storage and processing.

• Data visualization.

• Data governance and quality.

• Data marketplace solutions.

• Machine learning and AI capabilities.

The selected candidate will support Data Science and AI initiatives, particularly the execution, monitoring, and evaluation of Large Language Model (LLM) inference experiments.

Key responsibilities

LLM Inference Execution and Monitoring

• Execute batch inference jobs using Databricks and other environments.

• Monitor inference processes and troubleshoot execution issues.

• Maintain logs and report model anomalies.

• Escalate technical issues when required.

Pipeline Development and Evaluation Support

• Assist in running DSPy pipelines.

• Load and configure custom models and configurations.

• Support model evaluation using internal metrics.

• Track, analyze, and summarize inference experiment performance.

Data Preparation and Experimentation

• Prepare, clean, and organize legal text datasets for inference and evaluation.

• Annotate model outputs to support error analysis and benchmarking.

• Maintain reproducible scripts and notebooks for inference experiments.

  
  

Required technical skills

Skill area

Requirements

Programming

Python

Python libraries

pandas, scikit-learn, PyTorch

LLM frameworks

DSPy, LangChain, Hugging Face or equivalent

Data and ML platforms

Databricks, MLflow

Machine Learning

Model inference, evaluation and experimentation

Data handling

Dataset preparation, cleaning and annotation

Documentation

Reproducible scripts, notebooks and experiment tracking

Candidate qualifications

• Educational background in Computer Science, Data Science, Artificial Intelligence or Machine Learning.

• Students and recent graduates are welcome.

• Familiarity with Python and machine learning libraries.

• Exposure to LLM frameworks and model experimentation.

• Experience or familiarity with Databricks, MLflow or other cloud-based ML environments.

• Strong attention to detail and organizational skills.