Azure Data Engineer
- Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Posting language
- English
- Posted
Original job description
Key Responsibilities
- Design, develop, and maintain ETL/ELT pipelines using Azure Data Factory (ADF).
- Develop data transformation and processing solutions using Azure Databricks, PySpark, and Python.
- Write complex SQL queries, stored procedures, and performance-optimized scripts.
- Implement data ingestion and storage solutions using Azure Data Lake Storage Gen2 (ADLS).
- Build and maintain data warehouse solutions using Azure Synapse Analytics.
- Work with structured, semi-structured, and unstructured data.
- Implement incremental data loads, CDC, partitioning, and data transformation logic.
- Optimize Spark jobs, SQL queries, and data pipelines for performance and cost.
- Implement data quality checks, error handling, logging, and monitoring.
- Integrate data from multiple sources, including databases, APIs, files, and cloud platforms.
- Implement security and access controls using Azure Key Vault, Managed Identity, and RBAC.
- Collaborate with data architects, analysts, business teams, and other engineers.
- Use Git and Azure DevOps for version control, CI/CD, and deployment automation.
- Troubleshoot production issues and ensure data pipeline reliability.
Preferred Skills
- Experience with Azure Functions and Logic Apps.
- Knowledge of event-driven data processing using Event Hubs or Stream Analytics.
- Experience with Microsoft Fabric or Power BI integration.
- Understanding of data modeling, star schema, and snowflake schema.
- Familiarity with Azure Monitor, Log Analytics, and application monitoring.
- Knowledge of Agile/Scrum methodologies.
More about this job
Responsibilities
Design, build, and maintain Azure-based data pipelines, ingestion and storage solutions, and data warehouse systems, integrating data from databases, APIs, files, and cloud platforms. Optimize performance and cost, implement data quality, monitoring, security, and deployment practices, and collaborate with technical and business teams to ensure reliable production operations.
Requirements
The posting identifies entry-level seniority and describes work involving Azure data services, Python, PySpark, SQL, data integration, and pipeline optimization. Preferred skills include Azure Functions, Logic Apps, event-driven processing, Microsoft Fabric or Power BI, data modeling, monitoring tools, and Agile/Scrum; no education requirement is stated.
Skills
- Azure Data Factory
- Azure Databricks
- PySpark
- Python
- SQL
- Azure Data Lake Storage Gen2
- Azure Synapse Analytics
- ETL/ELT Pipelines
- Data Modeling
- Change Data Capture
- Data Quality
- Git
- Azure DevOps
- Azure Key Vault
- Managed Identity
- Role-Based Access Control
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
Keywords
- Azure Data Factory
- Azure Databricks
- PySpark
- Python
- SQL
- Azure Data Lake Storage Gen2
- Azure Synapse Analytics
- ETL/ELT
- Data Ingestion
- Data Warehousing
- Structured Data
- Semi-Structured Data
- Unstructured Data
- Incremental Loads
- Change Data Capture
- Partitioning
- Data Quality
- Azure Key Vault
- Managed Identity
- Role-Based Access Control
- Git
- Azure DevOps
- CI/CD
- Azure Functions
- Logic Apps
- Event Hubs
- Stream Analytics
- Microsoft Fabric
- Power BI
- Agile/Scrum