Role: Platform Engineer (Azure/Databricks)
Type: Contract (4 days on-site)
Location: Toronto
Job Summary
We are seeking a highly skilled Senior/Lead Platform Engineer with strong expertise in Azure, Databricks, Python, PySpark, and Spark SQL to design, develop, and support scalable cloud-based data platforms. The ideal candidate will have proven experience building robust data solutions, automating platform onboarding, implementing CI/CD pipelines, and resolving complex production issues in highly regulated environments.
Key Responsibilities
- Design, develop, and maintain cloud-native data platform solutions using Azure Databricks, Python, PySpark, and Spark SQL.
- Build and optimize data ingestion, transformation, and processing pipelines.
- Automate platform provisioning and deployment using Infrastructure as Code (IaC) and CI/CD practices.
- Develop secure, scalable, and performant applications and platform services.
- Troubleshoot, debug, and resolve complex development and production issues.
- Collaborate with business and technology stakeholders to gather requirements and deliver high-quality solutions.
- Ensure platform reliability, observability, and operational excellence.
- Follow industry best practices for security, compliance, and software engineering.
Required Skills
- Strong hands-on experience with: Azure Cloud, Azure Databricks, Python, PySpark, Spark SQL, CI/CD tools (Jenkins, GitHub)
- Strong coding, debugging, and problem-solving abilities.
- Experience working independently in complex technical environments.
Preferred Skills
- Terraform
- Dynatrace or other observability/monitoring tools
- Java
- Kafka
- Microservices architecture
- REST APIs
- Airflow
- Distributed storage platforms (ADLS, S3, Google Cloud Storage)
