Data Analyst - Expert
- Toronto, ON, Canada
Pay Rate: $89.88/hr -$103.89/hr · Full-time
Applications open in a new tab.
Job details
- Pay
- Pay Rate: $89.88/hr -$103.89/hr
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
- Location: Toronto, ON - Hybrid - (Thursday and 4th Friday of the month – may change due to business requirements)
- Pay Rate: $89.88/hr -$103.89/hr
- Contract Length: 6 Months with possibility for extension
Responsibilities
- Develop and maintain ADF pipelines and Databricks solutions.
- Build Python notebooks using Spark and implement Medallion Architecture.
- Configure Unity Catalog, data governance, security, and permissions.
- Migrate Hive Metastore to Unity Catalog.
- Design and manage Delta Lake tables and schemas.
- Implement CI/CD pipelines using Azure DevOps or GitHub Enterprise.
- Manage Azure resources, Entra ID, SSO, and RBAC.
- Monitor pipelines and optimize performance and costs.
- Troubleshoot technical issues and lead a small team.
- 7+ Years experience with ADF, Databricks and Azure Functions (C#)
- In depth development experience python/notebook on Databricks/Spark/panda
- Expert working knowledge of Unity Catalog and data governance setup/configuration
- Experience in loading datasets from various sources using ADF to Databricks with Medallion architecture
- Experience on converting Databricks workspace from hive metastore to unity catalog & setup associated permissions
- Creating delta table schema/tables with proper design/properties & security permissions
- DevOps & deployment pipeline using Azure DevOps/GitHub Enterprise for both ADF and databricks
- In depth experience on Azure Entra ID for SSO/RBAC setup
- In depth experience on working with/deploying to Azure PaaS resources such as Azure SQL PaaS, ADF/SHIR, ADLS, Databricks, Synapse dedicated SQL databases
- Familiar to work in lock down windows environment for highly regulated industry
- Ability to work independently and lead a small team.
- Proven ability to work independently with minimum supervision
- Ability to lead a small team and technical problem-solving sessions
- Excellent Communication skills
- Financial Services Industry experience is an Asset.
- 10+ total experience but for Databricks can be 5-8 years experience.
- ADF orchestrations with Databricks
- Experience with set up and configuration of data governance
- Databricks (Python notebooks, jobs, clusters, Delta Lake, medallion architecture, performance tuning/cost optimization)
- CI/CD, security
- Monitoring
- Previous Banking experience or experience in a financial institution.
- Nice to have experience on Delta Live Table.
Raise is an established hiring firm with over 65 years of experience. We believe strongly in making the world a better place through work, which is why we’re a certified B Corporation and donate 10% of our profits to charity.
We strive to build teams that reflect the diversity of the communities we work in. We encourage all qualified applicants to apply, including people from traditionally underrepresented groups such as women, visible minorities, Indigenous peoples, people identifying as LGBTQ2SI, veterans, and people with visible/nonvisible disabilities.
We have a dedicated webpage for accommodations where you can learn more about what we offer and request accommodation: https://raise.jobs/accommodations/
In order to submit candidates for roles, our clients will sometimes require personal information to confirm the identity of applicants and their legal status to work. Raise will never ask you for personal or banking information unless you have been selected for a job. If you are ever unsure about the legitimacy of this or another job posting by Raise (or have any other questions), please contact us at +1 800-567-9675 or hello@raiserecruiting.com
#FIN24
More about this job
Responsibilities
Develop and maintain Azure Data Factory pipelines and Databricks solutions, including Python/Spark notebooks, Medallion Architecture, and Delta Lake tables. Configure data governance, Unity Catalog, security, and CI/CD; manage Azure resources, monitor and optimize pipelines, troubleshoot issues, and lead a small team.
Requirements
Requires 10+ years of overall experience, including substantial experience with Azure Data Factory, Databricks, Azure Functions, Python, Spark, data governance, and Azure deployments. Candidates should have experience with Unity Catalog migration and permissions, CI/CD, security, monitoring, performance and cost optimization, and the ability to work independently and lead a small team; financial services experience is an asset.
Skills
- Azure Data Factory
- Databricks
- Azure Functions
- Python
- Apache Spark
- Unity Catalog
- Data Governance
- Medallion Architecture
- Delta Lake
- CI/CD
- Azure DevOps
- GitHub Enterprise
- Microsoft Entra ID
- Role-Based Access Control
- Performance Tuning
- Team Leadership
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Management & Leadership
- Finance & Accounting
Keywords
- Azure Data Factory
- Databricks
- Azure Functions
- C#
- Python
- Apache Spark
- Pandas
- Unity Catalog
- Hive Metastore
- Medallion Architecture
- Delta Lake
- Azure DevOps
- GitHub Enterprise
- Microsoft Entra ID
- Single Sign-On
- Role-Based Access Control
- Azure SQL
- Azure Data Lake Storage
- Azure Synapse Analytics
- Self-Hosted Integration Runtime
- CI/CD
- Data Governance
- Data Security
- Pipeline Monitoring
- Performance Tuning
- Cost Optimization
- Delta Live Tables
- Banking
- Financial Services