Junior Data Engineer – Azure Databricks / ADF / PySpark
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
Required Skills & Qualifications
- Degree or diploma in Computer Science, Software Engineering, Information Technology, or a related field.
- Hands-on experience developing data engineering solutions using Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), and Azure Databricks.
- Strong programming skills in Python, SQL, PySpark, and Apache Spark for data processing and transformation.
- Experience developing and maintaining ETL/ELT pipelines, data ingestion frameworks, and scalable batch or near real-time data processing solutions.
- Hands-on knowledge of Delta Lake, Lakehouse Architecture, and Medallion Architecture (Bronze, Silver, Gold).
- Experience developing Databricks notebooks, workflows, data processing jobs, and optimizing Delta Lake tables.
- Strong understanding of data validation, data quality, reconciliation, monitoring, and troubleshooting of data pipelines.
- Experience with Azure DevOps, Git, CI/CD pipelines, automated deployments, and version control.
- Knowledge of REST APIs, data integration patterns, automation frameworks, and reusable development utilities.
- Understanding of data governance, metadata management, data cataloguing, security, privacy, and access control.
- Experience with software development best practices, including code reviews, automated testing, documentation, and secure coding.
- Strong communication and collaboration skills, with experience working in Agile/Scrum teams alongside architects, data engineers, analysts, and business stakeholders.
More about this job
Responsibilities
Develop and maintain scalable data ingestion and ETL/ELT pipelines using Azure Data Factory, Azure Databricks, PySpark, and related Azure data services. Build and optimize Databricks jobs and Delta Lake tables, while supporting data quality, monitoring, deployment automation, and governance.
Requirements
Requires a degree or diploma in computer science, software engineering, information technology, or a related field, along with hands-on experience building data engineering solutions on Azure. Candidates should have strong Python, SQL, PySpark, and Spark skills, experience with data pipelines and Lakehouse patterns, and familiarity with CI/CD, data quality, governance, and Agile collaboration.
Skills
- Azure Data Factory
- Azure Data Lake Storage
- Azure Databricks
- Python
- SQL
- PySpark
- Apache Spark
- ETL/ELT Pipelines
- Delta Lake
- Lakehouse Architecture
- Medallion Architecture
- Azure DevOps
- Git
- CI/CD
- REST APIs
- Data Quality
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Consulting
Keywords
- Azure Data Factory
- Azure Data Lake Storage
- Azure Databricks
- Python
- SQL
- PySpark
- Apache Spark
- ETL/ELT
- Data Ingestion
- Batch Processing
- Near Real-Time Processing
- Delta Lake
- Lakehouse Architecture
- Medallion Architecture
- Databricks Notebooks
- Databricks Workflows
- Data Validation
- Data Quality
- Data Reconciliation
- Azure DevOps
- Git
- CI/CD
- REST APIs
- Data Governance
- Metadata Management
- Data Cataloguing
- Access Control
- Automated Testing
- Secure Coding
- Agile/Scrum