Azure Data Engineer
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
- Strong hands-on experience with Azure Data Factory (ADF), Azure Databricks, Python, PySpark, SQL, and Azure cloud services.
- Expertise in developing scalable data ingestion, transformation, and orchestration pipelines using Databricks and Apache Airflow.
- Experience implementing Medallion Architecture (Bronze, Silver, Gold), Delta Lake, and reusable data engineering frameworks.
- Develop reusable Databricks notebooks, data transformation components, orchestration workflows, and automated data quality checks.
- Strong experience with Azure DevOps, Git, CI/CD pipelines, and automated deployment processes.
- Experience with Spark performance tuning, pipeline troubleshooting, data reconciliation, and workload optimization.
- Knowledge of data governance, metadata management, Unity Catalog, and enterprise data engineering best practices.
More about this job
Responsibilities
Develop scalable data ingestion, transformation, and orchestration pipelines using Azure Databricks, Apache Airflow, and related Azure services. Build reusable engineering components and notebooks, implement automated data quality checks, and optimize and troubleshoot data workloads.
Requirements
Requires strong hands-on experience with Azure Data Factory, Azure Databricks, Python, PySpark, SQL, and Azure cloud services. Candidates should also have experience with Medallion Architecture, Delta Lake, Azure DevOps, Git, CI/CD, Spark optimization, data governance, metadata management, and Unity Catalog.
Skills
- Azure Data Factory
- Azure Databricks
- Python
- PySpark
- SQL
- Azure Cloud Services
- Apache Airflow
- Medallion Architecture
- Delta Lake
- Data Engineering Frameworks
- Data Quality
- Azure DevOps
- Git
- CI/CD
- Spark Performance Tuning
- Unity Catalog
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
Keywords
- Azure Data Factory
- Azure Databricks
- Python
- PySpark
- SQL
- Azure Cloud Services
- Data Ingestion
- Data Transformation
- Pipeline Orchestration
- Apache Airflow
- Medallion Architecture
- Bronze Layer
- Silver Layer
- Gold Layer
- Delta Lake
- Reusable Data Engineering Frameworks
- Databricks Notebooks
- Data Quality Checks
- Azure DevOps
- Git
- CI/CD Pipelines
- Automated Deployment
- Spark Performance Tuning
- Pipeline Troubleshooting
- Data Reconciliation
- Workload Optimization
- Data Governance
- Metadata Management
- Unity Catalog
- Enterprise Data Engineering