Data Engineer
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
Data Engineer (Contract, LLM Exposure a Plus)
Overview
We're looking for a Data Engineer to join us on a 6+ month contract. You'll design, build, and own data pipelines and platforms for our clients, from ingestion and transformation through to warehousing and validation. Some projects also bring LLMs into the mix, so you'll get hands-on exposure to applying them alongside solid data engineering. The work spans a range of client needs, so expect to build new systems from scratch as well as improve existing ones.
Responsibilities
- Own end-to-end development, testing, and validation frameworks for the platforms you build.
- Build scalable products from scratch, anticipating how users will break them and designing for future issues, not just today's cases.
- Build infrastructure as code with Terraform on AWS services such as Redshift and S3.
- Develop data transformations using dbt and SQL.
- Write Python scripts for pipeline automation and validation.
- Validate outputs with test cases and AI-agent-assisted testing.
- Monitor platforms and deliver new development after launch.
Technical Requirements
- Solid foundation in data engineering, including designing and building end-to-end data pipelines.
- Strong understanding of data processing (batch and incremental) and data storage (warehouses, data lakes, data modeling).
- Strong experience building data pipelines with Python and SQL.
- Hands-on AWS experience, especially Redshift and S3.
- Terraform for infrastructure as code.
- dbt for data transformation.
- Experience writing automated test cases and validation frameworks.
- Familiarity with LLMs, including when to use deterministic versus non-deterministic (LLM) approaches.
Why Join Us
We move fast and give people real ownership. You'll work in a high-ownership environment where your decisions shape the product from day one, with a small team that values initiative, clear thinking, and getting things shipped. If you enjoy building from scratch, solving hard problems end to end, and seeing your work used by real clients, we'd love to hear from you.
More about this job
Responsibilities
Design, build, test, and maintain end-to-end data pipelines and platforms, including ingestion, transformation, warehousing, and validation. Build AWS infrastructure with Terraform, develop transformations using dbt and SQL, automate and validate pipelines with Python, and monitor and improve platforms after launch.
Requirements
Requires a solid data engineering foundation, experience designing end-to-end pipelines, and strong Python and SQL skills. Candidates should have hands-on experience with AWS Redshift and S3, Terraform, dbt, automated testing and validation, and familiarity with LLMs and when to use deterministic versus non-deterministic approaches.
Skills
- Data Engineering
- Data Pipeline Development
- Python
- SQL
- AWS
- Amazon Redshift
- Amazon S3
- Terraform
- dbt
- Data Transformation
- Data Warehousing
- Data Modeling
- Automated Testing
- Data Validation
- LLMs
- Infrastructure as Code
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Engineering
Keywords
- Data Engineering
- Data Pipelines
- Data Ingestion
- Data Transformation
- Batch Processing
- Incremental Processing
- Data Warehousing
- Data Lakes
- Data Modeling
- Python
- SQL
- AWS
- Amazon Redshift
- Amazon S3
- Terraform
- Infrastructure as Code
- dbt
- Automated Testing
- Validation Frameworks
- LLMs
- AI-Agent-Assisted Testing
- Pipeline Automation
- Platform Monitoring
- Contract
- Artificial Intelligence