Data Engineer
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
- Strong hands-on experience with Azure Data Factory (ADF), Azure Databricks, Databricks Jobs, Workflows, and Notebooks.
- Expertise in PySpark, Python, SQL, Delta Lake, and ETL/ELT pipeline development.
- Experience implementing Medallion Architecture (Bronze, Silver, Gold) for enterprise data platforms.
- Develop reusable data ingestion, transformation, and orchestration pipelines.
- Implement automated data quality checks, reconciliation, testing, and performance optimization.
- Experience with Azure DevOps, Git, and CI/CD integration.
- Good to have: Unity Catalog, Databricks Asset Bundles, and Lakeflow Spark Declarative Pipelines.
More about this job
Responsibilities
Develop reusable data ingestion, transformation, and orchestration pipelines using Azure and Databricks technologies. Implement data quality checks, reconciliation, testing, and performance optimization for enterprise data platforms.
Requirements
Requires strong hands-on experience with Azure Data Factory, Azure Databricks, and Databricks Jobs, Workflows, and Notebooks, along with expertise in PySpark, Python, SQL, Delta Lake, and ETL/ELT development. Experience with Medallion Architecture, Azure DevOps, Git, and CI/CD is expected; Unity Catalog, Databricks Asset Bundles, and Lakeflow Spark Declarative Pipelines are advantageous.
Skills
- Azure Data Factory
- Azure Databricks
- Databricks Jobs
- Databricks Workflows
- Databricks Notebooks
- PySpark
- Python
- SQL
- Delta Lake
- ETL/ELT Pipeline Development
- Medallion Architecture
- Data Ingestion
- Data Transformation
- Data Orchestration
- Azure DevOps
- CI/CD
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Consulting
Keywords
- Azure Data Factory
- Azure Databricks
- Databricks Jobs
- Databricks Workflows
- Databricks Notebooks
- PySpark
- Python
- SQL
- Delta Lake
- ETL/ELT
- Medallion Architecture
- Bronze Layer
- Silver Layer
- Gold Layer
- Data Ingestion
- Data Transformation
- Data Orchestration
- Data Quality Checks
- Data Reconciliation
- Testing
- Performance Optimization
- Azure DevOps
- Git
- CI/CD
- Unity Catalog
- Databricks Asset Bundles
- Lakeflow Spark Declarative Pipelines
- Enterprise Data Platforms