Principal Azure Databricks Architect
- Toronto, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
Job Title: Principal Azure Databricks Architect
Location: Toronto, ON (hybrid)
Duration: long term contract
Rate: CAD110-CAD120/hr
Role Overview
We are seeking an experienced Principal Azure Databricks Architect to design, implement, and optimize enterprise-scale data platforms within the Azure ecosystem. The ideal candidate will have deep expertise in Azure Databricks, enterprise data lakes, Lakehouse architecture, data engineering, cloud cost optimization, and modern CI/CD practices.
This role requires strong hands-on coding skills, architectural leadership, and the ability to work independently with business and technical stakeholders to deliver scalable, secure, and cost-effective data solutions.
Key Responsibilities
- Design, architect, and implement enterprise-scale data lakes and Lakehouse solutions on Azure.
- Lead the setup and optimization of Azure Databricks and Azure Data Lake Storage environments.
- Develop scalable data pipelines using Azure Data Factory, Databricks, Python, PySpark, and SQL.
- Design robust data models and integration frameworks for structured and unstructured data.
- Write complex, production-ready code and troubleshoot application, pipeline, and performance issues.
- Establish CI/CD processes using Jenkins and GitHub.
- Implement monitoring and observability solutions using Dynatrace.
- Provide architectural guidance around Lakehouse design, Databricks environments, governance, security, and performance.
- Drive Databricks and cloud FinOps initiatives to optimize platform usage and costs.
- Work with relational and NoSQL databases to support enterprise data integration requirements.
- Collaborate with stakeholders to understand business needs and independently deliver robust technical solutions.
- Provide technical leadership, establish engineering best practices, and guide development teams.
- Evaluate emerging technologies related to AI agents, skills, and Model Context Protocol, also known as MCP.
Required Qualifications
- Extensive experience designing and implementing enterprise data lakes and Lakehouse architectures.
- Strong hands-on expertise in:
- Azure Databricks
- Azure Data Factory
- Azure Data Lake Storage
- Python and PySpark
- Advanced SQL
- Jenkins and GitHub
- Dynatrace or similar observability tools
- Strong knowledge of Databricks FinOps, cloud cost optimization, and platform performance.
- Experience working with relational and NoSQL databases.
- Strong programming experience in Python, PySpark, or Java.
- Ability to develop complex code and independently diagnose and resolve technical issues.
- Excellent analytical and problem-solving capabilities.
- Strong stakeholder management and communication skills.
- Ability to translate business requirements into scalable technical architecture.
Nice-to-Have Skills
- Experience developing dashboards and reports using Power BI.
- Exposure to one or more BI tools, such as Cognos, Power BI, Sigma, or Strategy.
- Experience building AI agents and related AI capabilities.
- Understanding of Model Context Protocol, or MCP.
- Additional hands-on experience with Dynatrace.
- Functional experience in retail banking, capital markets, or securities processing.
More about this job
Responsibilities
Architect and implement enterprise-scale Azure data lakes and Lakehouse platforms, including Databricks environments, data pipelines, integrations, and production-ready code. Provide technical leadership on governance, security, performance, observability, CI/CD, and cost optimization while collaborating with stakeholders and evaluating emerging AI technologies.
Requirements
Requires extensive experience designing enterprise data lakes and Lakehouse architectures, with strong hands-on skills in Azure Databricks, Azure Data Factory, Azure Data Lake Storage, Python or PySpark, SQL, CI/CD, and observability tools. Candidates should also bring cloud FinOps and database integration expertise, strong problem-solving and communication skills, and the ability to independently deliver scalable technical solutions.
Skills
- Azure Databricks
- Azure Data Factory
- Azure Data Lake Storage
- Lakehouse Architecture
- Data Lake Architecture
- Python
- PySpark
- Advanced SQL
- Jenkins
- GitHub
- Dynatrace
- Cloud Cost Optimization
- Databricks FinOps
- Data Engineering
- Relational and NoSQL Databases
- CI/CD
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
- Management & Leadership
- Finance & Accounting
Keywords
- Azure Databricks
- Azure Data Factory
- Azure Data Lake Storage
- Lakehouse Architecture
- Enterprise Data Lakes
- Data Engineering
- Data Pipelines
- Python
- PySpark
- SQL
- Java
- Jenkins
- GitHub
- CI/CD
- Dynatrace
- Observability
- Databricks FinOps
- Cloud Cost Optimization
- Data Modeling
- Data Integration
- Relational Databases
- NoSQL Databases
- Data Governance
- Cloud Security
- Power BI
- Cognos
- Sigma
- Strategy
- AI Agents
- Model Context Protocol