TPM- Data Engineering

Technical Program Manager (TPM) – Databricks Platform

Role Overview

We are seeking an experienced Technical Program Manager (TPM) with strong expertise in Databricks and modern data platforms to lead and drive large-scale data & AI transformation programs. This role will work at the intersection of business, data engineering, and platform architecture, ensuring successful delivery of enterprise-grade data solutions on Databricks.

Key Responsibilities

Program Leadership

• Drive end-to-end execution of data platform programs built on Databricks (Lakehouse architecture).
  
• Manage multiple cross-functional teams including data engineering, data science, DevOps, and BI teams.
  
• Own program roadmap, delivery timelines, risk management, and stakeholder communication.
  
• Establish governance frameworks for program delivery, quality, and performance tracking.
  

Databricks Platform Ownership

• Lead design and implementation of Databricks-based data platforms (Azure Databricks / AWS Databricks / GCP).
  
• Ensure best practices in:
  • Delta Lake
    
  • Data pipelines (batch + streaming)
    
  • Medallion architecture (Bronze, Silver, Gold layers)
    
• Drive platform optimization including cost, performance, and scalability.
  

Technical Strategy & Execution

• Collaborate with architects to define data architecture standards and patterns.
  
• Oversee development of:
  • ETL/ELT pipelines
    
  • Data ingestion frameworks
    
  • Data transformation and curation layers
    
• Ensure integration with ecosystem tools like:
  • Azure Data Factory / Airflow
    
  • Power BI / Tableau
    
  • MLflow, Feature Store
    

Stakeholder & Business Alignment

• Act as a bridge between business leaders and engineering teams.
  
• Translate business requirements into technical execution plans.
  
• Provide leadership updates and executive reporting on program progress.
  

Delivery Excellence

• Define and track KPIs, SLAs, and program success metrics.
  
• Identify and mitigate risks, dependencies, and bottlenecks.
  
• Ensure adherence to Agile / Scrum / SAFe methodologies.
  

Data Governance & Security

• Ensure compliance with:
  • Data governance policies
    
  • Security and privacy regulations
    
• Implement best practices for:
  • Data lineage
    
  • Data quality
    
  • Access controls
    

Skills & Qualifications

Mandatory Skills

• 10+ years in technology roles, with 5+ years in program management.
  
• Strong hands-on or working knowledge of:
  • Databricks (must-have)
    
  • Apache Spark / PySpark / SQL
    
  • Delta Lake architecture
    
• Experience in cloud platforms:
  • Azure (preferred), AWS, or GCP
    
• Proven experience delivering enterprise data platform programs.
  

Technical Expertise

• Data Engineering concepts:
  • Data modeling, ETL/ELT, pipelines
    
• Real-time streaming (Kafka / Event Hub)
  
• CI/CD for data pipelines
  
• DataOps / MLOps frameworks
  

Program Management Skills

• Strong experience in:
  • Agile / Scrum / SAFe frameworks
    
  • Program governance and reporting
    
• Excellent stakeholder management skills, including C-level engagement.