Our client is running a modern data platform built on AWS and actively moving through a major Databricks implementation. The core objective is transforming data into actionable business products for marketing and promotions. The project team needs a seasoned Senior Data Engineer to help lead the charge, stabilize key deliverables, and guide our technical strategy. This is a critical engagement on a high-visibility, fast-paced project where both strong technical hands-on capability and sharp soft skills are essential.
This is a 6 month contract based in North Toronto and requires 5 days onsite.
Key Responsibilities:
- Lead the design, development, and operation of scalable, cloud-based data platforms supporting analytics and machine learning workloads.
- Design and maintain high-quality data pipelines (ETL/ELT) with strong standards for reliability, observability, metadata, and lineage.
- Build and operate Databricks-based data engineering solutions, including Spark, SparkSQL, and Delta Lake pipelines.
- Enable and support MLOps workflows, including data pipelines, feature engineering, and integration with AWS SageMaker for model training, deployment, and monitoring.
- Collaborate closely with Data Science and ML Engineering teams to transition models from experimentation to production.
- Work with Data Governance and Data Management teams to ensure compliance with enterprise standards and audit requirements.
- Implement and maintain data quality frameworks, monitoring, and automated validation checks.
- Define and maintain enterprise data assets, data models, and transformation logic.
- Perform advanced data analysis to troubleshoot complex data issues and production incidents.
- Contribute to CI/CD pipelines for data and analytics workloads, including testing and deployment automation.
- Provide technical mentorship to data engineers, promoting engineering best practices and code quality.
- Evaluate and recommend tools and technologies for data lineage, observability, and platform scalability.
- Partner with architecture and platform teams to define long-term data and ML platform strategy.
- Produce clear documentation and contribute to shared engineering standards and knowledge bases.
Required Skills & Qualifications:
- 5+ years of professional experience in data engineering or data platform development.
- Advanced proficiency in Python and SQL.
- Strong experience building data platforms on AWS, including services such as Lambda, Glue, Redshift, Step Functions, CloudFormation, Athena, and related services.
- Hands-on experience with Databricks, including Spark, Delta Lake, notebook-driven development, and production pipelines.
- Experience enabling or supporting MLOps, including integration with AWS SageMaker for model lifecycle management.
- Experience delivering CI/CD pipelines for data and analytics systems.
- Strong understanding of data modeling (relational, dimensional) and performance optimization.
- Experience implementing data quality, monitoring, and observability solutions.
- Proficient with source control tools such as Git and collaborative development practices.
- Working knowledge of Linux and scripting.
- Experience with file formats (CSV, JSON, XML) and diverse data integration patterns.
- Strong understanding of cloud-native architectures and emerging data technologies.
- Experience managing technical debt, refactoring, and evolving systems over time.
- Ability to balance short-term delivery with long-term architectural considerations.
- Experience as a DBA or with Salesforce (SFDC) is an asset.
- Bachelor’s degree in Computer Science, Engineering, or equivalent practical experience.
