Senior Data Engineer
- Toronto, ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
- Strong hands-on experience building and operating AWS-based data platforms and data pipelines.
- Advanced expertise with Apache Spark, PySpark, distributed data processing, and performance optimization.
- Experience with modern data lake technologies, particularly Apache Hudi and Apache Iceberg.
- Strong Python programming and software engineering practices, including testing, code quality, and Git-based development workflows.
- Hands-on experience with Apache Airflow or comparable workflow orchestration platforms.
- Experience with Terraform, CI/CD pipelines, and infrastructure automation.
- Ability to independently translate business requirements into production-ready data solutions.
More about this job
Responsibilities
Build and operate AWS-based data platforms and production data pipelines, using distributed processing and modern data lake technologies. Translate business requirements independently into production-ready data solutions and support them with workflow orchestration, infrastructure automation, and CI/CD.
Requirements
Requires strong hands-on experience with AWS data platforms, Apache Spark and PySpark, modern data lake technologies, and Python. Also requires software engineering practices, workflow orchestration experience, Terraform, CI/CD, and the ability to independently deliver production-ready solutions.
Skills
- AWS
- Data Platforms
- Data Pipelines
- Apache Spark
- PySpark
- Distributed Data Processing
- Performance Optimization
- Apache Hudi
- Apache Iceberg
- Python
- Software Engineering
- Testing
- Git
- Apache Airflow
- Terraform
- CI/CD
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Engineering
- Consulting
Keywords
- AWS
- Data Platforms
- Data Pipelines
- Apache Spark
- PySpark
- Distributed Data Processing
- Performance Optimization
- Data Lakes
- Apache Hudi
- Apache Iceberg
- Python
- Software Engineering
- Testing
- Code Quality
- Git
- Workflow Orchestration
- Apache Airflow
- Terraform
- CI/CD
- Infrastructure Automation
- Production Data Solutions
- Business Requirements
- IT Services
- IT Consulting