Cloud Data Engineer
- Toronto, ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Experience
- Resource have knowledge in Bank and financial domain (Minimum 12+ years of experience).
- Posting language
- English
- Posted
Original job description
Resource have strong data warehouse technical knowledge.
Resource have knowledge in Bank and financial domain (Minimum 12+ years of experience).
Design, develop, and maintain scalable data pipelines and workflows on AWS
Build and manage data lakes and data warehouses (e.g., S3, Redshift)
Develop ETL/ELT processes using tools like AWS Glue, Lambda, or Spark
Strong solution Knowledge in AWS Cloud and hands on experience with ETL process (like Kafka message processing, batch interface data injection and other business layer process)
Optimize data ingestion, transformation, and loading processes
Work with stakeholders to understand and deliver data requirements
Ensure data quality, governance, and security compliance
Monitor and troubleshoot data pipelines and workflows
Automate deployments using CI/CD pipelines
Improve performance, cost optimization, and scalability
Coordinate with clients, data users and key stakeholders to understand feature requirements needed merge them to create reusable design patterns
Data onboarding using the developed frameworks
Understand and make sense of available code in Netezza to design a best way to implement its current features in AWS Data Lake
Unit test code and aid with QA/SIT/Perf testing
Migration to production environment
Good to have
AWS Certifications (Solutions Architect / Data Analytics)
Experience with Airflow (or Managed Workflows for Apache Airflow - MWAA)
Exposure to Snowflake, Databricks, or Delta Lake
Amazon S3 (Data Lake)
AWS Glue (ETL)
AWS Lambda
Amazon Redshift
AWS Step Functions
AWS IAM & CloudWatch
IBM Netezza
Resource have knowledge in Bank and financial domain (Minimum 12+ years of experience).
Design, develop, and maintain scalable data pipelines and workflows on AWS
Build and manage data lakes and data warehouses (e.g., S3, Redshift)
Develop ETL/ELT processes using tools like AWS Glue, Lambda, or Spark
Strong solution Knowledge in AWS Cloud and hands on experience with ETL process (like Kafka message processing, batch interface data injection and other business layer process)
Optimize data ingestion, transformation, and loading processes
Work with stakeholders to understand and deliver data requirements
Ensure data quality, governance, and security compliance
Monitor and troubleshoot data pipelines and workflows
Automate deployments using CI/CD pipelines
Improve performance, cost optimization, and scalability
Coordinate with clients, data users and key stakeholders to understand feature requirements needed merge them to create reusable design patterns
Data onboarding using the developed frameworks
Understand and make sense of available code in Netezza to design a best way to implement its current features in AWS Data Lake
Unit test code and aid with QA/SIT/Perf testing
Migration to production environment
Good to have
AWS Certifications (Solutions Architect / Data Analytics)
Experience with Airflow (or Managed Workflows for Apache Airflow - MWAA)
Exposure to Snowflake, Databricks, or Delta Lake
Amazon S3 (Data Lake)
AWS Glue (ETL)
AWS Lambda
Amazon Redshift
AWS Step Functions
AWS IAM & CloudWatch
IBM Netezza
More about this job
Responsibilities
Design, develop, and maintain scalable data pipelines and warehouses on AWS, specifically migrating features from Netezza to an AWS Data Lake. The role involves optimizing data ingestion and collaborating with stakeholders to deliver reusable design patterns and ensure data security.
Requirements
Requires over 12 years of experience with strong technical knowledge of data warehousing and the banking/financial domain. Proficiency in AWS services like Glue, Lambda, and Redshift is essential, with AWS certifications and experience in Airflow or Snowflake being preferred.
Skills
- AWS
- ETL/ELT
- Data Warehousing
- Python
- Spark
- AWS Glue
- Amazon Redshift
- Amazon S3
- AWS Lambda
- Kafka
- Airflow
- Netezza
- CI/CD
- Data Lake
- Data Governance
- Financial Domain Knowledge
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Consulting
- Finance & Accounting
Keywords
- AWS
- Cloud Data Engineer
- Data Warehouse
- Banking
- Financial Domain
- S3
- Redshift
- AWS Glue
- Lambda
- Spark
- Kafka
- Data Lake
- ETL
- ELT
- CI/CD
- Airflow
- MWAA
- Snowflake
- Databricks
- Delta Lake
- Step Functions
- IAM
- CloudWatch
- IBM Netezza
- Data Governance
- Data Onboarding
- Solution Architecture
- Data Analytics
- SIT
- Perf Testing