Position: Azure Data Support Analyst – Databricks & Data Factory
Location: Hybrid (4days/week)- Toronto, ON
Job type: Full-Time/Permanent Hiring
We are seeking an experienced Azure Data Support Analyst to support and troubleshoot Azure-based data platforms and production data pipelines. The role requires strong hands-on experience with Azure Data Factory, Azure Databricks, ADLS, Azure SQL, PySpark/Spark SQL, and Delta Lake, along with a solid understanding of data integration, ETL/ELT, and data warehousing.
The ideal candidate will be comfortable troubleshooting production pipeline failures, Databricks cluster issues, performance bottlenecks, and data-processing problems while working with DevOps and monitoring teams.
Key Responsibilities
- Provide production and operational support for Azure-based data platforms and data pipelines.
- Monitor and troubleshoot Azure Data Factory pipelines, data ingestion processes, and scheduled data workflows.
- Troubleshoot Azure Databricks notebooks, jobs, clusters, Spark workloads, and data-processing issues.
- Analyze and resolve pipeline failures, data processing errors, cluster issues, and performance bottlenecks.
- Work with PySpark and Spark SQL to investigate and troubleshoot data transformation and processing issues.
- Support data stored and processed through ADLS, Delta Lake, and Azure SQL.
- Troubleshoot ETL/ELT workflows, data ingestion patterns, transformations, and data warehouse processes.
- Analyze logs and monitoring information using Azure Monitor, Log Analytics, and Databricks logs.
- Work with development and engineering teams to identify root causes and resolve recurring production issues.
- Support Azure DevOps, Git, and CI/CD pipelines related to data platform deployments.
- Participate in incident, problem, and change management activities.
- Document troubleshooting procedures, recurring issues, root-cause findings, and operational runbooks.
- Identify opportunities to improve pipeline reliability, monitoring, performance, and operational support.
Required Skills
- Strong experience with Microsoft Azure data services.
- Hands-on experience with Azure Data Factory, Azure Databricks, ADLS, and Azure SQL.
- Hands-on experience with Databricks notebooks, PySpark/Spark SQL, and Delta Lake.
- Strong understanding of ETL/ELT, data ingestion, data transformation, and data warehousing concepts.
- Experience troubleshooting data pipeline failures, Databricks cluster issues, and performance bottlenecks.
- Experience with Azure DevOps, Git, and CI/CD pipelines.
- Familiarity with Azure Monitor, Log Analytics, and Databricks logging.
- Strong analytical and troubleshooting skills with the ability to investigate production data issues.
Preferred Experience
- Experience supporting enterprise-scale Azure data platforms.
- Experience with production incident and problem management.
- Experience with data pipeline monitoring and operational automation.
- Knowledge of Spark performance optimization and Databricks troubleshooting.
- Experience working in Agile/DevOps environments.
