Intermediate Software Developer
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
The client is undertaking a large-scale Data and Artificial Intelligence (AI) transformation initiative focused on developing and enabling the Ontario Data Platform (ODP). ODP is an enterprise technology ecosystem designed to support data ingestion, storage, processing, visualization, governance, data quality, and data marketplace capabilities. The selected candidate will support AI/ML initiatives within ODP, focusing on running and monitoring machine learning inference workloads, supporting Large Language Model (LLM) pipelines, evaluating model performance, and preparing datasets for experimentation. The role requires practical experience with Python, machine learning libraries, LLM frameworks, and cloud-based ML environments.
Key Responsibilities
Key Responsibilities
- Execute, monitor, and troubleshoot batch inference jobs in Databricks and other ML environments.
- Support DSPy pipelines, including custom model and configuration loading.
- Conduct LLM evaluation runs and track model performance metrics.
- Prepare, clean, and validate legal text datasets for inference and evaluation.
- Annotate model outputs for error analysis and benchmarking.
- Maintain reproducible Python scripts, notebooks, and experiment documentation.
- Identify, document, and escalate model anomalies.
- Background in Computer Science, Data Science, AI/ML, or a related field (students and recent graduates welcome).
- Hands-on experience with Python and libraries such as pandas, scikit-learn, or PyTorch.
- Exposure to LLM frameworks such as DSPy, LangChain, or Hugging Face.
- Familiarity with Databricks, MLflow, or equivalent cloud ML environments.
- Strong analytical, troubleshooting, organizational, and documentation skills.
- Hands-on experience building or executing DSPy-based LLM pipelines.
- Experience with batch inference, LLM evaluation, and experiment tracking.
- Knowledge of prompt optimization, model benchmarking, and error analysis.
- Experience processing unstructured text or legal datasets.
- Familiarity with ML workflow automation and reproducible experimentation.
More about this job
Responsibilities
Execute, monitor, and troubleshoot batch inference workloads and DSPy pipelines, and conduct LLM evaluations while tracking model performance. Prepare and annotate legal text datasets, maintain reproducible scripts and experiment documentation, and identify and escalate model anomalies.
Requirements
Candidates should have a background in computer science, data science, AI/ML, or a related field; students and recent graduates are welcome. Hands-on Python and machine learning library experience, exposure to LLM frameworks and cloud ML environments, and strong analytical, troubleshooting, organizational, and documentation skills are required.
Skills
- Python
- Pandas
- Scikit-Learn
- PyTorch
- DSPy
- LangChain
- Hugging Face
- Databricks
- MLflow
- Machine Learning
- LLM Evaluation
- Batch Inference
- Dataset Preparation
- Model Performance Tracking
- Error Analysis
- Experiment Documentation
Visa sponsorship
Not detected in the job text
Categories
- Software
- Technology
- Data & Analytics
- Science & Research
Keywords
- Python
- Pandas
- Scikit-Learn
- PyTorch
- DSPy
- LangChain
- Hugging Face
- Databricks
- MLflow
- Ontario Data Platform
- Artificial Intelligence
- Machine Learning
- Large Language Models
- Batch Inference
- LLM Pipelines
- Model Evaluation
- Model Benchmarking
- Prompt Optimization
- Experiment Tracking
- Dataset Preparation
- Legal Text
- Unstructured Text
- Data Ingestion
- Data Processing
- Data Governance
- Data Quality
- Reproducible Experimentation
- Workflow Automation
- Error Analysis
- Cloud ML Environments