Closed listing
Lead Azure Databricks Platform Engineer
- ON, Canada
- Ontario, Canada, Canada
Full-time
This opening has closed.
The original listing is kept below for reference.
Job details
- Job type
- Full-time
- Closing date
- Posting language
- English
- Posted
Similar jobs hiring now
Search current openingsOriginal job description
Archived listing. The details below describe a past opening.
Lead Azure Databricks Platform Engineer
Hybrid – Toronto ON
Contract Role
Key Responsibilities
We are seeking a seasoned Principal Cloud Engineer [Individual Contributor] who thrives in restricted Azure setups – where native features are locked down, so you’ll code workarounds like custom microservices to replicate them. Focus on automating Databrick workspace onboarding via IAM modules, using Terraform for IaC despite limitations, and build ingestion pipelines that bypass disabled tools with Python or similar. Need proven coding chops to trigger those behaviors securely, plus banking or fintech experience handling compliance in tricky environments.
Strong Expertise in Azure, Databricks, Python, Pyspark, SparkSQL, CI-CD (Jenkins, Github), Obs (Dynatrace),
Nice to have : Java, Kafka , Casandra, Microservices, API, Flink, ZeroMQ, Protocol Buffers
Able to write the complex code and fix the issues
Should have strong problem solving capability
Collaborate with stakeholders to understand requirements and deliver robust solutions independently
Responsible for both development and strong expertise to debug and fix any post production issues for their developed code
Functional experience of Retail Banking, capital markets, securities processing will be advantage
Must have skills – findable only
- Azure, Databricks, Python, Pyspark, SparkSQL, CI-CD (Jenkins, Github),
Must have skills that can be trained
- Obs (Dynatrace)
- Nice to have skills
- Terraform
- Java,
- Kafka ,
- Microservices,
- API
- Distributed Storage System (like ADLS, S3, Google Storage)
- Airflow
More about this job
Responsibilities
Lead development and automation of Azure Databricks platform onboarding, infrastructure, and ingestion pipelines, using secure custom solutions where native Azure features or tools are restricted. Collaborate with stakeholders, deliver solutions independently, and debug and resolve production issues in developed code.
Requirements
Requires strong expertise in Azure, Databricks, Python, PySpark, Spark SQL, and CI/CD using Jenkins or GitHub, along with strong coding, troubleshooting, and problem-solving abilities. Banking or fintech compliance experience is sought; retail banking, capital markets, and securities processing experience is an advantage, while Terraform and other listed technologies are additional assets.
Skills
- Azure
- Databricks
- Python
- PySpark
- Spark SQL
- CI/CD
- Jenkins
- GitHub
- Dynatrace
- Terraform
- Java
- Kafka
- Microservices
- APIs
- Distributed Storage Systems
- Airflow
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Software
- Data & Analytics
- Engineering
- Finance & Accounting
Keywords
- Azure
- Azure Databricks
- Python
- PySpark
- Spark SQL
- CI/CD
- Jenkins
- GitHub
- Dynatrace
- Terraform
- Java
- Kafka
- Cassandra
- Microservices
- APIs
- Flink
- ZeroMQ
- Protocol Buffers
- Distributed Storage Systems
- ADLS
- Amazon S3
- Google Cloud Storage
- Airflow
- Infrastructure As Code
- IAM
- Ingestion Pipelines
- Cloud Engineering
- Banking
- Fintech
- Compliance