Data Engineer
- Toronto, ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
- Strong hands-on experience with Azure Data Factory (ADF), Azure Databricks, Python, PySpark, SQL, and Azure cloud services.
- Expertise in developing scalable data ingestion, transformation, and orchestration pipelines using Databricks and Apache Airflow.
- Experience implementing Medallion Architecture (Bronze, Silver, Gold), Delta Lake, and reusable data engineering frameworks.
- Develop reusable Databricks notebooks, data transformation components, orchestration workflows, and automated data quality checks
- Strong experience with Azure DevOps, Git, CI/CD pipelines, and automated deployment processes.
- Experience with Spark performance tuning, pipeline troubleshooting, data reconciliation, and workload optimization.
- Knowledge of data governance, metadata management, Unity Catalog, and enterprise data engineering best practices.
- For Lead Engineers: Experience in solution design, technical architecture, code reviews, mentoring, and collaboration with client architects.
- Good to have: Databricks Asset Bundles, REST APIs, and integration with enterprise data platforms.
More about this job
Responsibilities
Develop scalable data ingestion, transformation, and orchestration pipelines, reusable data engineering components, and automated data quality checks. Optimize and troubleshoot data workloads, support governance and metadata management, and, for lead-level roles, contribute to architecture, code reviews, mentoring, and client collaboration.
Requirements
Requires strong hands-on experience with Azure Data Factory, Azure Databricks, Python, PySpark, SQL, Azure services, and pipeline orchestration. Candidates should also have experience with CI/CD, Spark optimization, data quality and reconciliation, governance, and enterprise data engineering practices; lead engineers additionally need architecture and mentoring experience.
Skills
- Azure Data Factory
- Azure Databricks
- Python
- PySpark
- SQL
- Azure Cloud Services
- Apache Airflow
- Medallion Architecture
- Delta Lake
- Data Engineering
- Azure DevOps
- Git
- CI/CD
- Spark Performance Tuning
- Data Governance
- Unity Catalog
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Engineering
Keywords
- Azure Data Factory
- Azure Databricks
- Python
- PySpark
- SQL
- Azure Cloud Services
- Data Ingestion
- Data Transformation
- Pipeline Orchestration
- Apache Airflow
- Medallion Architecture
- Bronze, Silver, Gold
- Delta Lake
- Reusable Data Engineering Frameworks
- Databricks Notebooks
- Data Quality Checks
- Azure DevOps
- Git
- CI/CD Pipelines
- Automated Deployment
- Spark Performance Tuning
- Pipeline Troubleshooting
- Data Reconciliation
- Workload Optimization
- Data Governance
- Metadata Management
- Unity Catalog
- Solution Design
- Technical Architecture
- Databricks Asset Bundles