At a glance
Automatically prepared from the listing. Check the original description for the full requirements.
Jump to the original descriptionResponsibilities
Design, develop, and maintain scalable data pipelines and warehouses on AWS, specifically migrating features from IBM Netezza to a Data Lake. The role involves optimizing data ingestion and collaborating with stakeholders to ensure data quality and security compliance.
Requirements
Requires strong technical knowledge of data warehousing and at least 5 years of experience in the banking and financial domain. Proficiency in AWS services like Glue, Lambda, and Redshift is essential for implementing ETL processes.
Working hours
40 hours per week
Skills
- Amazon S3
- AWS Glue
- AWS Lambda
- Amazon Redshift
- AWS Step Functions
- AWS IAM
- CloudWatch
- IBM Netezza
- ETL/ELT
- Apache Spark
- Kafka
- CI/CD
- Data Warehousing
- Data Lake
- Python
- AWS Cloud
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Consulting
- Finance & Accounting
Keywords
- AWS
- Data Engineer
- Data Warehouse
- Netezza Migration
- S3
- Glue
- Lambda
- Redshift
- Step Functions
- IAM
- CloudWatch
- ETL
- ELT
- Spark
- Kafka
- CI/CD
- Airflow
- MWAA
- Snowflake
- Databricks
- Delta Lake
- Banking
- Financial Domain
- Data Lake
- Solutions Architect
- Data Analytics
- Data Governance
- Data Onboarding
- Unit Testing
- Performance Tuning
Original job description
AWS Data Engineer – Data Warehouse & Netezza Migration
Location: Toronto; Hybrid 2 days a week in office
“MUST HAVE” skills and experience for this requirement.
Amazon S3 (Data Lake)
AWS Glue (ETL)
AWS Lambda
Amazon Redshift
AWS Step Functions
AWS IAM & CloudWatch
IBM Netezza
Role and Responsibilities
Resource have strong data warehouse technical knowledge.
Resource have knowledge in Bank and financial domain (Minimum 5+ years of experience).
Design, develop, and maintain scalable data pipelines and workflows on AWS Build and manage data lakes and data warehouses (e.g., S3, Redshift) Develop ETL/ELT processes using tools like AWS Glue, Lambda, or Spark Strong solution Knowledge in AWS Cloud and hands on experience with ETL process (like Kafka message processing, batch interface data injection and other business layer process) Optimize data ingestion, transformation, and loading processes Work with stakeholders to understand and deliver data requirements Ensure data quality, governance, and security compliance Monitor and troubleshoot data pipelines and workflows Automate deployments using CI/CD pipelines Improve performance, cost optimization, and scalability Coordinate with clients, data users and key stakeholders to understand feature requirements needed merge them to create reusable design patterns Data onboarding using the developed frameworks Understand and make sense of available code in Netezza to design a best way to implement its current features in AWS Data Lake Unit test code and aid with QA/SIT/Perf testing Migration to production environment
Good to have
AWS Certifications (Solutions Architect / Data Analytics) Experience with Airflow (or Managed Workflows for Apache Airflow - MWAA) Exposure to Snowflake, Databricks, or Delta Lake
