We’re looking for a **Senior Databricks Data Engineer / Solutions Architect** to own an **Azure Databricks Lakehouse** from data ingestion and engineering through governed data products, **Power BI**, and **AI/RAG** solutions.
Key Responsibilities
- Architect **Medallion Lakehouse** solutions using Azure Databricks, Delta Lake & Unity Catalog
- Build scalable ETL/ELT pipelines using **PySpark, Spark SQL & ADF**, including CDC and data validation
- Design **dimensional models, data marts & Power BI** solutions
- Integrate data from databases, **REST APIs, SharePoint & SFTP**
- Build **GenAI/RAG** solutions using Azure OpenAI, Azure AI Search, FastAPI & Streamlit
- Develop **Azure DevOps CI/CD** pipelines and support Terraform across environments
- Develop, schedule & monitor **Python jobs on Databricks**
- Tune **Azure SQL / SQL Server** for performance, security & reliability
- Implement data quality, lineage, governance & **RBAC**
- Lead migrations from **on-prem SQL Server, SSIS & SSRS**
Required Skills
🔹 **8+ years** in Data Engineering / BI with **3+ years of Azure Databricks**
🔹 Strong **Databricks, Spark SQL, T-SQL & query optimization**
🔹 **Python, PySpark, SQL/T-SQL, Delta Lake**
🔹 **ADF, ADLS Gen2, Azure SQL & Synapse**
🔹 Medallion Architecture & Dimensional Modeling
🔹 **Power BI – DAX, Power Query & RLS**
🔹 Azure DevOps CI/CD
🔹 Strong database administration, performance tuning & security experience
🔹 Excellent stakeholder communication
Nice to Have: Databricks Data Engineer Professional, DP-203, AZ-305, Microsoft Fabric/OneLake, Terraform, **RAG/Vector Search/Embeddings**, or public-sector/regulated-environment experience.
