Senior Data Engineer (AWS, Big Data, ETL, Netezza, Spark, Kafka, Data Warehousing)
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Experience
- 10+ years of experience in Data Engineering / Big Data / Platform Engineering.
- Posting language
- English
- Posted
Original job description
Senior Data Engineer | AWS Cloud | Big Data | ETL | Netezza | Spark | Kafka | Data Platform Modernization
Job Summary
The ideal candidate will have strong expertise in designing, developing, and supporting scalable data pipelines and distributed systems, along with hands-on experience in Big Data ecosystem tools, AWS services, and real-time data processing.
This role involves working on data platform modernization, cloud migrations, Data Warehousing, and ETL in a fast-paced enterprise environment.
Required Technical Skills
Cloud Technologies
- Strong experience in AWS Cloud services:
- EMR
- EC2
- S3
- VPC
- RDS
- Redshift
- AWS Glue
- IAM
- CloudWatch
- CloudFormation
- Experience with:
- Airflow (or AWS Managed Workflows)
Databases
- Experience working with:
- Netezza (Mandatory)
- SQL-based systems:
- SQL Server
- PostgreSQL
- Data warehouses:
- Teradata
- Redshift
- Netezza
ETL Tools
- Hands-on experience with:
- SSIS
- Pentaho or similar ETL tools
Programming & Scripting
- Strong proficiency in:
- SQL
- Shell scripting
- Good to have:
- Python
Operating Systems
- Strong experience in Unix/Linux environments.
Key Qualifications
- 10+ years of experience in Data Engineering / Big Data / Platform Engineering.
Key Responsibilities
- Design, develop, maintain, and support scalable data pipelines using Big Data and AWS technologies.
- Lead and support data platform migration initiatives (On-Prem to AWS Cloud), ideally with Netezza background.
- Develop and manage ETL/ELT processes using tools like SSIS, Pentaho, or similar.
- Implement and manage AWS services such as:
- EMR
- S3
- EC2
- Redshift
- Glue
- Airflow
- Build and optimize data workflows and orchestration pipelines using Airflow.
- Work with real-time streaming technologies such as:
- Kafka
- Spark Streaming
- Perform data ingestion, transformation, and validation from multiple data sources.
- Optimize SQL queries for performance and scalability.
- Monitor system performance, troubleshoot issues, and ensure system reliability.
- Collaborate with cross-functional teams including:
- Developers
- Architects
- Infrastructure teams.
More about this job
Responsibilities
The senior data engineer at Astra North Infoteck will build and maintain data pipelines on AWS and big data tools, and lead the move of data platforms from on-premises systems to the cloud. The role also covers ETL processes, query tuning, monitoring and working with developers, architects and infrastructure teams.
Requirements
Candidates need 10+ years in data engineering, big data or platform engineering, with mandatory Netezza experience and strong AWS skills. They also need SQL, shell scripting, Unix/Linux and hands-on ETL tool experience.
Skills
- AWS EMR
- AWS EC2
- AWS S3
- AWS VPC
- AWS RDS
- AWS Redshift
- AWS Glue
- AWS IAM
- AWS CloudWatch
- AWS CloudFormation
- Apache Airflow
- AWS Managed Workflows
- Netezza
- SQL Server
- PostgreSQL
- Teradata
- SSIS
- Pentaho
- SQL
- Shell scripting
- Python
- Unix/Linux
- Apache Kafka
- Spark Streaming
- Apache Spark
Categories
- Technology
- Data & Analytics
- Software
Keywords
- Senior Data Engineer
- AWS
- Amazon EMR
- Amazon S3
- Amazon EC2
- Amazon Redshift
- AWS Glue
- Airflow
- Netezza
- Spark
- Spark Streaming
- Kafka
- ETL
- ELT
- SSIS
- Pentaho
- Data Warehousing
- Big Data
- SQL
- Shell scripting
Seniority
Senior
Tools
- Netezza
- AWS EMR
- AWS S3
- AWS EC2
- AWS Redshift
- AWS Glue
- Airflow
- Kafka
- Spark
- SSIS
Languages
- English