AWS Data Engineer
- Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Posting language
- English
- Posted
Original job description
Responsibilities:
- Designed and developed scalable AWS Data Lake solutions for ingesting, processing, and analyzing enterprise data.
- Built and maintained ETL/ELT pipelines using AWS Glue, PySpark, and AWS Lambda.
- Developed batch and real time data pipelines integrating multiple source systems into Amazon S3 data lakes.
- Implemented data transformation, cleansing, and enrichment processes using AWS Glue Jobs and Glue Workflows.
- Managed metadata catalogs using AWS Glue Data Catalog and optimized data discovery capabilities.
- Designed data ingestion frameworks leveraging S3, Lambda, EventBridge, and Step Functions.
- Created analytical datasets for reporting and business intelligence consumption.
- Optimized Glue jobs for performance, scalability, and cost efficiency.
- Implemented data quality, governance, and monitoring controls across pipelines.
- Worked with Redshift, Athena, and Lake Formation to support data analytics and governance initiatives.
- Developed CI/CD processes for data pipelines and automated deployment across environments.
- Collaborated with business stakeholders, application teams, and data architects to define data integration requirements.
Required Skills:
- AWS Glue
- AWS Data Lake
- Amazon S3
- PySpark, Python
- Athena, Redshift
- Lambda, Step Functions
- SQL & Data Modeling
- ETL/ELT Development
- Lake Formation
- Data Governance & Data Quality
- Event Bridge and Data Pipeline Automation
More about this job
Responsibilities
Design and maintain scalable AWS data lakes and batch and real-time ETL/ELT pipelines, including data ingestion, transformation, quality controls, and metadata management. Optimize data pipelines and support analytics and governance using AWS services, while automating deployments and collaborating with stakeholders to define integration requirements.
Requirements
The role requires experience with AWS Glue, S3, PySpark, Python, Athena, Redshift, Lambda, Step Functions, SQL, and data modeling. It also calls for ETL/ELT development, Lake Formation, data governance and quality, EventBridge, and data pipeline automation.
Skills
- AWS Glue
- AWS Data Lake
- Amazon S3
- PySpark
- Python
- Amazon Athena
- Amazon Redshift
- AWS Lambda
- AWS Step Functions
- SQL
- Data Modeling
- ETL/ELT Development
- AWS Lake Formation
- Data Governance
- Data Quality
- Amazon EventBridge
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
Keywords
- AWS Glue
- AWS Data Lake
- Amazon S3
- PySpark
- Python
- Amazon Athena
- Amazon Redshift
- AWS Lambda
- AWS Step Functions
- SQL
- Data Modeling
- ETL/ELT
- AWS Lake Formation
- Data Governance
- Data Quality
- Amazon EventBridge
- Data Pipeline Automation
- AWS Glue Data Catalog
- AWS Glue Workflows
- Data Ingestion
- Data Transformation
- Data Cleansing
- Data Enrichment
- Batch Data Pipelines
- Real-Time Data Pipelines
- CI/CD
- Data Analytics
- Business Intelligence
- Metadata Management
- Cost Optimization