We are seeking an experienced Azure Databricks Data Architect to lead enterprise-scale data architecture, data engineering, analytics, AI, and cloud modernization initiatives.
The ideal candidate will bring extensive experience designing and implementing enterprise data platforms using Microsoft Azure and Databricks, with strong expertise in Lakehouse architecture, data governance, and AI-enabled analytics.
This role requires a combination of strategic architecture leadership, hands-on technical expertise, and client-facing consulting experience to deliver scalable, secure, and high-performance enterprise data solutions.
Key Responsibilities
- Define end-to-end architecture for enterprise data, analytics, and AI platforms leveraging Azure and Databricks.
- Design scalable and secure Lakehouse architectures using Azure Data Services, Azure Databricks, Delta Lake, and Unity Catalog.
- Lead enterprise data platform modernization, migration, and cloud transformation initiatives.
- Architect data ingestion, integration, ETL/ELT, batch, and streaming data pipelines.
- Design AI-enabled analytics solutions using Databricks Genie, semantic models, and natural language analytics.
- Establish enterprise data governance, security, metadata management, lineage, and observability frameworks.
- Define architecture standards, reusable frameworks, reference architectures, and DataOps best practices.
- Drive platform performance optimization, scalability, reliability, and cost efficiency.
- Lead architecture reviews, HLD/LLD preparation, NFR assessments, and production readiness activities.
- Collaborate with business stakeholders, data engineers, AI specialists, and technology leadership teams.
- Support client workshops, technical presentations, solutioning, RFP responses, and effort estimations.
- Provide technical leadership and mentorship to architects and engineering teams.
Required Technical Skills & Experience
- 14+ years of experience in Enterprise Data Architecture, Data Engineering, Azure Data Platforms, and Cloud Modernization.
- Strong expertise in Microsoft Azure-based enterprise data architecture and Azure Databricks.
- Hands-on experience with Databricks Genie, Unity Catalog, Delta Lake, and Lakehouse architectures.
- Strong understanding of the Databricks Data Intelligence Platform, including Databricks SQL, Workflows, Lakeflow, MLflow, and Serverless Compute.
- Expertise in Azure Data Services, Python, PySpark, and SQL.
- Proven experience designing and implementing enterprise-scale Lakehouse architectures, data warehouses, and modern analytics platforms.
- Strong understanding of ETL/ELT, batch and streaming pipelines, APIs, orchestration, and DataOps.
- Experience with data modeling, semantic layers, data products, metadata management, data lineage, and data quality frameworks.
- Knowledge of enterprise security and governance, including IAM, RBAC, encryption, data masking, privacy, and compliance.
- Experience with architecture governance, HLD/LLD, NFR assessments, migration planning, and technical solution design.
- Good understanding of BFSI regulatory, security, privacy, audit, and resilience requirements.
Preferred Qualifications
- Experience implementing AI-enabled analytics and enterprise AI solutions using Databricks Genie.
- Experience with natural language analytics, semantic models, and governed data access frameworks.
- Azure Solutions Architect or Azure Data Engineering certifications.
- Databricks certifications in Data Engineering, Architecture, or related areas.
- Experience with enterprise architecture frameworks and large-scale cloud transformation programs.
Leadership & Consulting Skills
- Strong stakeholder management and client-facing consulting experience.
- Excellent communication and presentation skills.
- Ability to lead cross-functional and distributed architecture and engineering teams.
- Experience mentoring architects and engineers.
- Strong problem-solving, decision-making, and technical governance capabilities.
- Ability to manage risks, dependencies, and multiple stakeholder expectations.