Closed listing
Azure Data Engineer
- Canada
Full-time
This opening has closed.
The original listing is kept below for reference.
Job details
- Job type
- Full-time
- Posting language
- English
- Posted
Similar jobs hiring now
Search current openingsOriginal job description
Archived listing. The details below describe a past opening.
Key Responsibilities
- Design and develop scalable data pipelines using Azure services.
- Build and maintain ETL/ELT workflows for structured and unstructured data.
- Develop data integration solutions using Azure Data Factory (ADF).
- Store and process data using Azure Data Lake Storage (ADLS).
- Develop and optimize data transformations using Azure Databricks and PySpark.
- Design and maintain data warehouses using Azure Synapse Analytics.
- Write optimized SQL queries, stored procedures, and data transformations.
- Perform data cleansing, validation, and quality checks.
- Integrate data from multiple sources such as databases, APIs, files, and cloud applications.
- Monitor pipeline performance and troubleshoot data-processing failures.
- Implement data security, access control, and governance practices.
- Optimize data pipelines and cloud resources for performance and cost.
- Collaborate with data analysts, data scientists, software engineers, and business stakeholders.
- Maintain technical documentation for data pipelines, data models, and architecture.
Required Technical Skills
Azure
- Azure Data Factory (ADF)
- Azure Data Lake Storage Gen2
- Azure Databricks
- Azure Synapse Analytics
- Azure Key Vault
- Azure Monitor
- Azure Storage
- Azure SQL Database
Programming & Data
- Strong SQL
- Python
- PySpark
- ETL/ELT concepts
- Data transformation and data cleansing
- Data modeling
- Data warehousing concepts
Database Technologies
- SQL Server
- Azure SQL
- PostgreSQL/MySQL
- Relational and dimensional databases
Data Engineering Concepts
- ETL/ELT pipeline development
- Batch and incremental data processing
- Star and snowflake schemas
- Fact and dimension tables
- Slowly Changing Dimensions (SCD)
- Data partitioning
- Data quality and validation
- Performance optimization
- Data lake and data warehouse architecture
DevOps & Version Control
- Git/GitHub/Azure Repos
- CI/CD pipelines
- Azure DevOps
- Infrastructure and deployment automation
- Environment management across development, testing, and production
Preferred Skills
- Microsoft Fabric
- Azure Event Hubs
- Azure Stream Analytics
- Azure Functions
- Power BI integration
- Delta Lake
- Apache Spark
- Kafka
- Terraform
- Cloud security and governance
- Microsoft Purview
More about this job
Responsibilities
Design, build, and optimize scalable Azure data pipelines and ETL/ELT workflows, integrating data from multiple sources and supporting data lakes and warehouses. Ensure data quality, security, governance, and performance while monitoring pipelines, documenting architecture, and collaborating with technical and business teams.
Requirements
The role requires practical knowledge of Azure data services, including Data Factory, Data Lake Storage Gen2, Databricks, and Synapse Analytics, along with strong SQL, Python, and PySpark skills. Candidates should understand data engineering, modeling, warehousing, pipeline processing, quality validation, performance optimization, and DevOps practices.
Skills
- Azure Data Factory
- Azure Data Lake Storage Gen2
- Azure Databricks
- Azure Synapse Analytics
- SQL
- Python
- PySpark
- ETL/ELT
- Data Modeling
- Data Warehousing
- Data Quality and Validation
- Azure DevOps
- CI/CD
- Git
- Data Pipeline Optimization
- Cloud Security and Governance
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
Keywords
- Azure
- Azure Data Factory
- Azure Data Lake Storage Gen2
- Azure Databricks
- Azure Synapse Analytics
- Azure Key Vault
- Azure Monitor
- Azure Storage
- Azure SQL Database
- SQL Server
- PostgreSQL
- MySQL
- SQL
- Python
- PySpark
- ETL/ELT
- Data Pipelines
- Data Modeling
- Data Warehousing
- Dimensional Modeling
- Slowly Changing Dimensions
- Data Partitioning
- Data Quality
- Git
- GitHub
- Azure Repos
- Azure DevOps
- CI/CD
- Microsoft Fabric
- Delta Lake