Responsibilities
Support Azure Databricks production workloads by troubleshooting pipelines, jobs, workflows, Spark execution, clusters, performance bottlenecks, and data processing errors. Automate operational tasks and pipeline recovery, monitor systems, manage incidents, investigate root causes, and prepare runbooks and troubleshooting documentation.
Requirements
Requires strong hands-on experience supporting Azure Databricks in production, including pipeline troubleshooting and monitoring, along with expertise in Databricks Jobs and Workflows, Spark, Azure Data Factory, and SQL. Requires experience with Python and shell scripting, Azure Monitor, Log Analytics, incident management, and root-cause analysis; ServiceNow, SRE practices, SLI/SLO concepts, and Azure DevOps are desirable.
