Data Engineer
- Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Posting language
- English
- Posted
Original job description
Key Responsibilities
- Design, develop, and maintain scalable data pipelines.
- Build and manage ETL/ELT processes for data extraction, transformation, and loading.
- Collect, process, and transform structured and unstructured data from multiple sources.
- Develop and optimize SQL queries for efficient data processing.
- Design, implement, and maintain data warehouses and data lakes.
- Work with large datasets to ensure data accuracy, quality, and consistency.
- Develop automated workflows for data integration and processing.
- Monitor data pipelines and troubleshoot performance issues.
- Collaborate with data analysts, data scientists, and software development teams.
- Implement data validation, security, and governance practices.
- Maintain technical documentation and follow data engineering best practices.
Required Technical Skills
- Strong knowledge of Python and SQL.
- Understanding of ETL and ELT processes.
- Experience with databases such as MySQL, PostgreSQL, SQL Server, or Oracle.
- Knowledge of data warehousing concepts and data modeling.
- Familiarity with Apache Spark and PySpark.
- Understanding of Apache Airflow and workflow orchestration.
- Knowledge of data integration and pipeline development.
- Familiarity with cloud platforms such as AWS, Microsoft Azure, or Google Cloud.
- Understanding of Git and version control systems.
- Knowledge of Linux and basic shell scripting.
Required Soft Skills
- Strong analytical and problem-solving abilities.
- Good communication and collaboration skills.
- Attention to detail and data accuracy.
- Ability to manage multiple tasks and meet deadlines.
- Willingness to learn new tools and technologies.
Preferred Qualifications
- Experience working with big data technologies.
- Knowledge of Snowflake, Databricks, or Amazon Redshift.
- Familiarity with Apache Kafka and real-time data processing.
- Understanding of CI/CD and DevOps practices.
- Knowledge of Agile and Scrum methodologies.
- Experience developing and deploying cloud-based data pipelines.
More about this job
Responsibilities
Design, build, and maintain scalable data pipelines, ETL/ELT workflows, data warehouses, and data lakes while ensuring data quality, accuracy, security, and governance. Monitor and optimize data processing, troubleshoot pipeline issues, document technical practices, and collaborate with analysts, scientists, and software development teams.
Requirements
Requires strong Python and SQL skills, knowledge of ETL/ELT, databases, data warehousing and modeling, and familiarity with Spark, Airflow, cloud platforms, Git, Linux, and shell scripting. The role also calls for analytical problem-solving, communication, collaboration, attention to detail, and the ability to manage tasks and deadlines; big data, streaming, CI/CD, DevOps, Agile, and cloud pipeline experience are preferred.
Skills
- Python
- SQL
- ETL/ELT
- Data Pipelines
- Data Warehousing
- Data Modeling
- Apache Spark
- PySpark
- Apache Airflow
- Cloud Platforms
- Database Management
- Data Integration
- Git
- Linux
- Data Governance
- Problem-Solving
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
Keywords
- Python
- SQL
- ETL
- ELT
- Data Pipelines
- Data Integration
- Data Warehousing
- Data Lakes
- Data Modeling
- MySQL
- PostgreSQL
- SQL Server
- Oracle
- Apache Spark
- PySpark
- Apache Airflow
- AWS
- Microsoft Azure
- Google Cloud
- Git
- Linux
- Shell Scripting
- Big Data
- Snowflake
- Databricks
- Amazon Redshift
- Apache Kafka
- Real-Time Data Processing
- CI/CD
- DevOps