We are seeking a highly experienced Senior Databricks Data Engineer to support enterprise-scale data transformation and modernization initiatives for a leading consulting client.
This is a senior, hands-on technical role requiring someone who can operate independently while also providing technical leadership to the broader data engineering team. The ideal candidate will bring deep Databricks expertise, strong modern data engineering fundamentals, and the ability to drive technical decisions, review code, establish best practices, and mentor and train other engineers.
The successful candidate should be comfortable taking ownership of complex engineering initiatives with minimal oversight and serving as a senior technical resource for the team.
Key Responsibilities
- Design, build, optimize, and maintain scalable enterprise data pipelines and data engineering solutions using Databricks.
- Serve as a senior technical resource within the data engineering team, helping drive key technical and architectural decisions.
- Provide technical leadership, mentorship, and training to other data engineers.
- Conduct code reviews and help establish development standards, engineering best practices, and reusable architectural patterns.
- Take independent ownership of complex data engineering deliverables from design through implementation.
- Support the continued evolution and maturity of enterprise Databricks platforms.
- Develop and optimize ETL/ELT workflows across modern cloud data environments.
- Partner with architects, product leads, platform owners, governance, quality, and other technical stakeholders to deliver scalable data solutions.
- Contribute to data engineering standards, governance practices, performance optimization, and platform best practices.
- Support enterprise data transformation and modernization initiatives across analytics, reporting, cloud, and emerging AI-enabled data capabilities.
- Leverage emerging AI technologies and tooling to enhance data engineering capabilities and delivery, while remaining primarily focused on core data engineering.
- Collaborate effectively across technical and business teams in a fast-paced consulting and enterprise delivery environment.
Required Qualifications
- 5–8+ years of hands-on data engineering experience, with demonstrated experience operating at a senior or lead level.
- Approximately 3–5+ years of meaningful hands-on Databricks experience, with the technical depth required to guide other engineers and establish best practices.
- Strong hands-on experience designing, developing, and optimizing enterprise-scale data pipelines.
- Strong proficiency with SQL and Python.
- Experience working with modern cloud-based data platforms and technologies such as Databricks, Azure Data Factory, Azure Synapse, Snowflake, or comparable tools.
- Strong understanding of modern data architecture, ETL/ELT patterns, performance optimization, and data engineering best practices.
- Demonstrated ability to make technical decisions and work independently with minimal oversight.
- Experience conducting code reviews and providing technical guidance to other engineers.
- Ability to mentor, train, and upskill team members, particularly around Databricks and modern data engineering practices.
- Ability to contribute to architectural decisions while remaining highly hands-on technically.
- Strong communication and stakeholder collaboration skills.
Cloud Environment
Deep Databricks expertise is more important than experience with one specific hyperscaler. Candidates with Databricks experience across Azure, AWS, or GCP can be considered.
Experience within the Azure ecosystem remains valuable, including technologies such as:
- Azure Data Factory (ADF)
- Azure Synapse
- Azure Databricks
- Other Azure data and analytics services
However, Azure-specific experience is not mandatory for an otherwise strong Databricks engineer.
Nice-to-Have / Preferred
- Experience with SAP HANA to Databricks migrations.
- Experience with Snowflake, Azure Synapse, ADF, or other modern cloud data platforms.
- Exposure to newer Databricks capabilities and tooling, including MLflow and Agent Bricks.
- Exposure to AI technologies and experience applying AI-assisted capabilities within data engineering workflows.
- Experience supporting AI/ML or predictive analytics initiatives.
- Familiarity with Power BI and modern reporting environments.
- Experience across multiple hyperscalers or with cloud-agnostic Databricks architectures.
- Consulting or large-scale enterprise transformation experience.
Ideal Candidate Profile
This person should be a Data Engineer first. While exposure to AI and emerging technologies is valuable, this is not an AI Engineering role.
The ideal candidate is a highly capable senior Databricks engineer who can come into an established team and quickly become a trusted technical leader - someone who can build independently, make decisions, review the work of others, promote Databricks best practices, and train and mentor the broader team without requiring significant hand-holding