Job Details:
- Role: Senior Databricks Architect
- Require Domain exp: Insurance Domain experience is good to have.
- Location: North York, ON, Canada
- Work Model: Hybrid - 3 days in a week work from office is must, this is not initiate remote or complete remote role, find and share local candidates only who can commute easily.
- Onsite Requirement: Candidates must be available to work from the office from Day 1
- Job Type: Subcontractor (Initial Short Term and it may extend on client demand).
- Rate: CAD 70/hr All Inclusive HST and All should be included in the rate. Birlasoft will not provide any relocation assistance or any travelling/ accommodation or neither any addition expenses apart from discussed/agreed rate.
Mandatory Skills
#
Skill
Requirement
1
Databricks Data Engineering
Must have hands-on experience delivering at least one end-to-end Databricks implementation
2
Data Transformation & Pipeline Development
Experience building and managing transformation pipelines in Databricks
3
AWS Cloud Platform
Strong understanding of the AWS ecosystem; client environment is predominantly AWS
4
AWS Data Services
Working knowledge of S3, Redshift, Lake Formation, etc.
5
Apache Spark / PySpark
Strong data engineering expertise, preferably within Databricks
6
Strong Data Engineering Experience
Practical implementation experience, not just theoretical or presentation-level exposure
7
Problem-Solving & Delivery Mindset
Ability to work across technologies and drive solutions through successful delivery
Key Technical Experience:
We are looking for a Senior Databricks Architect with strong hands-on experience in:
- Databricks Data Engineering with at least one end-to-end Databricks implementation
- Data Transformation and Pipeline Development
- AWS Cloud Platform and AWS data services
- Amazon S3, Redshift, Lake Formation, etc.
- Apache Spark / PySpark
- Enterprise Data Engineering
- Data Lakes, Data Warehouses and Lakehouse architectures
- Problem-solving and end-to-end solution delivery
Preferred Experience:
- 15+ years of experience in Data & Analytics
- 5+ years of Databricks Architecture experience
- Experience with Databricks Unity Catalog, Delta Lake, Mosaic AI, Genie and Agent Bricks
- Experience in AI/GenAI, Lakehouse, Data Modernization and Advanced Analytics
- Strong consulting and stakeholder management experience
- AWS Solutions Architect certification is preferred
Role Overview:
The candidate will lead the architecture and implementation of enterprise-scale AWS Databricks data platforms, including Data Lakes, Data Warehouses and Lakehouse solutions supporting Analytics, AI and Machine Learning initiatives.
Key Responsibilities
- Design and implement secure, scalable and highly available cloud-based solutions on AWS.
- Lead the architecture, design and implementation of enterprise-scale data platforms leveraging Databricks on AWS.
- Define data architecture strategy, governance frameworks, release planning and platform roadmaps.
- Partner closely with customer stakeholders to drive data and technology modernization initiatives.
- Provide technical leadership, customer-facing communication and solution ownership from strategy through implementation.
- Implement Databricks Agent Bricks to enable domain-specific AI agents using enterprise data.
- Design and implement scalable Data Lake, Data Warehouse and Lakehouse architectures.
- Leverage Databricks Genie and AI/BI capabilities to enable natural-language analytics and self-service insights.
- Develop large-scale data engineering and analytics solutions using Databricks, AWS and PySpark.
- Integrate Amazon S3, AWS Lake Formation, AWS DevOps pipelines and Databricks workflows for end-to-end data and AI platform automation.
- Integrate structured, semi-structured and unstructured data sources into enterprise Data Lakes and Data Warehouses.
- Apply best practices in data modeling, data architecture, metadata management, data governance and data quality.
- Evaluate and communicate the advantages and trade-offs of different cloud, analytics and AI platforms.
- Translate business requirements into scalable AWS-based data and AI solutions.
- Define and implement AWS cloud governance, security, compliance and operational best practices.
- Identify opportunities to automate platform operations, deployments, monitoring and data engineering workflows.
- Lead architecture reviews, establish technical standards, mentor engineering teams and ensure successful delivery of enterprise-scale AWS Databricks programs.
Preferred Experience:
- 12+ years in Data & Analytics with 5+ years of Databricks Architecture experience
- Databricks Unity Catalog, Delta Lake, Mosaic AI, Genie and Agent Bricks
- AWS Solutions Architect certification
- AI/GenAI, Lakehouse, Data Modernization and Advanced Analytics experience
- Strong consulting and stakeholder management experience
Please prioritize candidates who have strong hands-on Databricks + AWS experience and are comfortable working onsite 3 days per week from Day 1.
Please share relevant profiles at the earliest.
