Azure Data Engineer
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Experience
- 7+ Years experience with ADF, Databricks and Azure Functions (C#)
- Posting language
- English
- Posted
Original job description
🔍 WE'RE HIRING: Sr Azure Data Engineer
📍 Location: Toronto, ON – Hybrid (in the office Thursday and 4th Friday of the month)
📅 Duration: 6+ Months with Possible extension
Job Description:
- 7+ Years experience with ADF, Databricks and Azure Functions (C#)
- In depth development experience python/notebook on Databricks/Spark/panda
- Expert working knowledge of Unity Catalog and data governance setup/configuration
- Experience in loading datasets from various sources using ADF to Databricks with Medallion architecture
- Experience on converting Databricks workspace from hive metastore to unity catalog & setup associated permissions
- Creating delta table schema/tables with proper design/properties & security permissions
- Nice to have experience on Delta Live Table.
- DevOps & deployment pipeline using Azure DevOps/GitHub Enterprise for both ADF and databricks
- In depth experience on Azure Entra ID for SSO/RBAC setup
- In depth experience on working with/deploying to Azure PaaS resources such as Azure SQL PaaS, ADF/SHIR, ADLS, Databricks, Synapse dedicated SQL databases
Must Have Requirements:
•ADF orchestrations with Databricks
- Experience with set up and configuration of data governance
•Databricks (Python notebooks, jobs, clusters, Delta Lake, medallion architecture, performance tuning/cost optimization)
•CI/CD, security
•Monitoring
More about this job
Responsibilities
Build and orchestrate data pipelines using Azure Data Factory and Databricks, loading data from multiple sources into a medallion architecture and designing secure Delta tables. Configure Unity Catalog and data governance, migrate workspaces from Hive Metastore, and support deployment, monitoring, security, and optimization across Azure data services.
Requirements
Requires 7+ years of experience with Azure Data Factory, Databricks, and Azure Functions, including strong Python, notebook, Spark, and Pandas development skills. Candidates must have hands-on expertise in Unity Catalog and data governance, Azure identity and access controls, CI/CD, monitoring, and Azure PaaS deployments.
Skills
- Azure Data Factory
- Databricks
- Azure Functions
- Python
- Apache Spark
- Pandas
- Unity Catalog
- Data Governance
- Medallion Architecture
- Delta Lake
- Azure DevOps
- GitHub Enterprise
- Microsoft Entra ID
- Role-Based Access Control
- CI/CD
- Performance Tuning
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
Keywords
- Azure Data Factory
- Databricks
- Azure Functions
- C#
- Python
- Notebooks
- Apache Spark
- Pandas
- Unity Catalog
- Data Governance
- Medallion Architecture
- Hive Metastore
- Delta Tables
- Delta Lake
- Delta Live Tables
- Azure DevOps
- GitHub Enterprise
- CI/CD
- Microsoft Entra ID
- Single Sign-On
- Role-Based Access Control
- Azure SQL
- Self-Hosted Integration Runtime
- Azure Data Lake Storage
- Azure Synapse Analytics
- Performance Tuning
- Cost Optimization
- Monitoring
- Security