We are looking for experience Data Engineers to join a fast growing company within the Financial Sector.
If you have strong experience with Databricks please take a look!!
About the Role
We are looking for a Databricks Data Engineer on a Full Time basis to help build and maintain their data pipelines and lakehouse infrastructure. This is a hands on role that involves taking ownership of moderately complex data engineering tasks, working closely with senior engineers on architecture decisions and partnering with analysts and data scientists to deliver reliable, well modelled data.
Key Responsibilities
- Build and maintain ETL/ELT pipelines using Databricks, Apache Spark and Delta Lake
- Develop and support bronze, silver and gold layer data models within a Lakehouse architecture
- Create and manage jobs and workflows using Lakeflow Jobs and Spark Declarative Pipelines (formerly Delta Live Tables)
- Write clean, efficient PySpark and SQL code for data transformation and processing
- Optimise Spark job performance through partitioning, caching and cluster configuration
- Implement data quality checks and basic monitoring/alerting for pipeline health
- Support data governance and access management using Unity Catalog, including permissions, lineage and catalog/schema structure
- Participate in code reviews, follow CI/CD practices and manage deployments using Declarative Automation Bundles (formerly Databricks Asset Bundles)
- Troubleshoot pipeline failures and data quality issues, escalating complex problems as needed
- Document pipelines, data models and processes for team and stakeholder reference
- Collaborate with analysts and data scientists to understand data requirements and deliver fit for purpose datasets
Technical Skills Required
- 5 to 7 years of experience in data engineering, including at least 2 to 4 years working directly with Databricks
- Strong practical PySpark and SQL skills for data transformation
- Understanding of Delta Lake and medallion (bronze/silver/gold) architecture
- Familiarity with the Databricks platform, including Lakeflow, declarative pipeline based ETL tooling, job orchestration and Unity Catalog
- Experience with at least one cloud platform (AWS, Azure or GCP)
- Experience with Git based version control and CI/CD workflows using Databricks Asset Bundles
- Understanding of data modelling and ETL/ELT design principles
- Exposure to or interest in AI assisted workflows on the platform, such as setting up Genie spaces or building simple agents with Agent Bricks
Desirable Skills
- Databricks Certified Data Engineer Associate or Professional
- Experience with Spark Declarative Pipelines (or DLT) and Unity Catalog
- Familiarity with orchestration tools such as Lakeflow Jobs, Airflow or Autosys
- Basic understanding of streaming data concepts, including Structured Streaming
- Experience with infrastructure as code (Terraform)
- Degree or diploma in Computer Science, Engineering or a related field, or equivalent practical experience
Salary
CAD $115,000 to $150,000 Depending on Experience
Next Steps
If this sounds like a good match for you please submit your CV by clicking apply. If you have any questions please reach out directly.
