Lead Data Engineer
- Calgary, AB, Canada
$50 to $53 CAD / hour · Contract
Applications open in a new tab.
Job details
- Pay
- $50 to $53 CAD / hour
- Job type
- Contract
- Apply by
- Work location
- Work Model: Hybrid (3 days per week in office: Monday, Tuesday, Thursday)
- Experience
- Experience Required: 6-8 Years
- Posting language
- English
- Posted
Original job description
Role: Databricks Lead (Lead Data Engineer)
Experience Required: 6-8 Years
Location: Edmonton / Calgary / Toronto (Any of these location)
Work Model: Hybrid (3 days per week in office: Monday, Tuesday, Thursday)
Contract Duration: 12 months
Role Summary
Seeking a Databricks Lead with strong expertise in Databricks to design, build, and operationalize scalable Supply Chain data products that deliver measurable business value. This role combines advanced analytics, data pipeline development, and product delivery to transform complex Supply Chain Management (SCM) data into trusted, governed, and actionable insights.
Key Responsibilities
Develop analytics solutions that address Supply Chain business challenges, leveraging knowledge of oil & gas operations, procurement, supplier performance, spend analytics, and enterprise value drivers.
Design, build, test, and optimize scalable ETL/ELT pipelines in Databricks using Python, PySpark, SQL, notebooks, jobs, and orchestration patterns to support reliable Silver and Gold data products.
Transform and model data from ERP and enterprise platforms, including Oracle Cloud, SAP, ADW, Microsoft Dataflow, Dataverse, and other approved data sources.
Build and evaluate machine learning, statistical, and AI-assisted solutions, including:
- Classification
- Forecasting
- Anomaly Detection
- Text Analytics
- Natural Language Processing (NLP)
Utilize Databricks platform capabilities such as:
- Notebooks
- Jobs
- Workflows
- Unity Catalog
- SQL Warehouses
- Genie Code
- Lakeflow / Lakeflow Designer
Maintain project documentation, including idea assessments, requirements, design documents, and weekly status reports.
Communicate technical concepts, requirements, solution designs, and product details effectively to business stakeholders in clear, non-technical language.
Required Skills:
Databricks
Python
PySpark
SQL / Oracle SQL
Data Architecture & Data Modeling
ETL / ELT Pipeline Development
Machine Learning
Supply Chain Analytics
Data Integration and Data Transformation
Preferred Domain Experience:
Supply Chain Management (SCM)
Oil & Gas Industry
Procurement Analytics
Supplier Performance Analytics
Spend Analytics
More about this job
Responsibilities
Design, build, test, and optimize scalable Databricks data pipelines and governed Silver and Gold data products using enterprise data sources. Develop analytics and machine learning solutions for supply chain challenges, document project work, and communicate technical designs and product details to business stakeholders.
Requirements
Requires 6–8 years of experience and expertise in Databricks, Python, PySpark, SQL, data architecture and modeling, ETL/ELT development, machine learning, and data integration. Supply chain management experience is preferred, particularly in oil and gas, procurement, supplier performance, and spend analytics.
Skills
- Databricks
- Python
- PySpark
- SQL
- Oracle SQL
- Data Architecture
- Data Modeling
- ETL/ELT Pipeline Development
- Machine Learning
- Supply Chain Analytics
- Data Integration
- Data Transformation
- Procurement Analytics
- Supplier Performance Analytics
- Spend Analytics
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Energy
- Logistics
Keywords
- Databricks
- Python
- PySpark
- SQL
- Oracle SQL
- ETL/ELT
- Data Architecture
- Data Modeling
- Machine Learning
- Supply Chain Management
- Oil And Gas
- Procurement
- Supplier Performance
- Spend Analytics
- Classification
- Forecasting
- Anomaly Detection
- Text Analytics
- Natural Language Processing
- Notebooks
- Jobs
- Workflows
- Unity Catalog
- SQL Warehouses
- Genie Code
- Lakeflow
- Oracle Cloud
- SAP
- ADW
- Microsoft Dataflow