Data Engineer
- Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Posting language
- English
- Posted
Original job description
Key Responsibilities
- Design, develop, and maintain ETL/ELT data pipelines.
- Extract, transform, and load data from multiple sources.
- Build and optimize data warehouses, data lakes, and databases.
- Write complex SQL queries and develop data transformation scripts using Python.
- Work with big data technologies such as Apache Spark and Hadoop.
- Implement data quality checks, validation, and monitoring.
- Optimize pipeline performance, reliability, and scalability.
- Collaborate with data analysts, data scientists, and business teams.
- Ensure data security, governance, and compliance.
- Troubleshoot data pipeline failures and resolve data-related issues.
Required Skills
- Programming: Python, SQL
- Databases: MySQL, PostgreSQL, SQL Server
- ETL Tools: Informatica, Talend, or dbt
- Big Data: Apache Spark, Hadoop, Kafka
- Cloud Platforms: AWS, Azure, or Google Cloud Platform
- Data Warehousing: Snowflake, Amazon Redshift, or Google BigQuery
- Workflow Orchestration: Apache Airflow
- Version Control: Git and GitHub
Preferred Skills
- Experience building cloud-based data pipelines.
- Knowledge of data modeling and dimensional modeling.
- Understanding of distributed computing and big data architecture.
- Familiarity with CI/CD, Docker, and data pipeline testing.
- Strong analytical and problem-solving skills.
More about this job
Responsibilities
Design, develop, and maintain ETL/ELT pipelines, data warehouses, data lakes, and databases, using SQL, Python, and big data technologies. Ensure data quality, security, governance, performance, and reliability while collaborating with analysts, data scientists, and business teams and resolving pipeline issues.
Requirements
The role requires programming skills in Python and SQL, experience with databases, ETL tools, big data technologies, cloud platforms, data warehouses, and workflow orchestration. Preferred qualifications include cloud-based pipeline experience, data and dimensional modeling, distributed computing knowledge, familiarity with CI/CD, Docker, and pipeline testing, and strong analytical and problem-solving skills.
Skills
- Python
- SQL
- MySQL
- PostgreSQL
- SQL Server
- Informatica
- Talend
- dbt
- Apache Spark
- Hadoop
- Kafka
- AWS
- Azure
- Google Cloud Platform
- Snowflake
- Apache Airflow
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
Keywords
- ETL
- ELT
- Data Pipelines
- Data Warehousing
- Data Lakes
- Databases
- Python
- SQL
- MySQL
- PostgreSQL
- SQL Server
- Informatica
- Talend
- dbt
- Apache Spark
- Hadoop
- Kafka
- AWS
- Azure
- Google Cloud Platform
- Snowflake
- Amazon Redshift
- Google BigQuery
- Apache Airflow
- Git
- GitHub
- Data Quality
- Data Modeling
- CI/CD
- Docker