Azure Data Engineer
- Toronto, ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
Role: Azure Data Engineer – Data Services Engineering (Transformation & Orchestration)
Location: Toronto, Ontario
Duration: Fulltime/ Contract
Work Type: Hybrid: (3 days office, 2 days remote)
- Strong hands-on experience with Azure Data Factory (ADF), Azure Databricks, Python, PySpark, SQL, and Azure cloud services.
- Expertise in developing scalable data ingestion, transformation, and orchestration pipelines using Databricks and Apache Airflow.
- Experience implementing Medallion Architecture (Bronze, Silver, Gold), Delta Lake, and reusable data engineering frameworks.
- Develop reusable Databricks notebooks, data transformation components, orchestration workflows, and automated data quality checks.
- Strong experience with Azure DevOps, Git, CI/CD pipelines, and automated deployment processes.
- Experience with Spark performance tuning, pipeline troubleshooting, data reconciliation, and workload optimization.
- Knowledge of data governance, metadata management, Unity Catalog, and enterprise data engineering best practices.
- For Lead Engineers: Experience in solution design, technical architecture, code reviews, mentoring, and collaboration with client architects.
- Good to have: Databricks Asset Bundles, REST APIs, and integration with enterprise data platforms.
More about this job
Responsibilities
Develop scalable data ingestion, transformation, and orchestration pipelines using Azure Databricks and Apache Airflow. Build reusable engineering components and notebooks, implement automated data quality checks, and optimize and troubleshoot pipelines.
Requirements
Requires hands-on experience with Azure Data Factory, Azure Databricks, Python, PySpark, SQL, and Azure cloud services, along with data pipeline development and orchestration expertise. Candidates should also have experience with CI/CD, Git, performance tuning, data governance, and metadata management; lead engineers additionally need solution design, code review, mentoring, and architecture collaboration experience.
Skills
- Azure Data Factory
- Azure Databricks
- Python
- PySpark
- SQL
- Azure Cloud Services
- Apache Airflow
- Medallion Architecture
- Delta Lake
- Data Engineering
- Azure DevOps
- Git
- CI/CD
- Spark Performance Tuning
- Unity Catalog
- Data Governance
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Engineering
Keywords
- Azure Data Factory
- Azure Databricks
- Python
- PySpark
- SQL
- Azure Cloud Services
- Apache Airflow
- Data Ingestion
- Data Transformation
- Data Orchestration
- Medallion Architecture
- Bronze Layer
- Silver Layer
- Gold Layer
- Delta Lake
- Reusable Data Engineering Frameworks
- Databricks Notebooks
- Data Quality Checks
- Azure DevOps
- Git
- CI/CD Pipelines
- Automated Deployment
- Spark Performance Tuning
- Pipeline Troubleshooting
- Data Reconciliation
- Workload Optimization
- Data Governance
- Metadata Management
- Unity Catalog
- Databricks Asset Bundles