Closed listing
Senior/Lead Azure Cloud Data Engineer
- Toronto, ON, Canada
Contract
This opening has closed.
The original listing is kept below for reference.
Job details
- Job type
- Contract
- Closing date
- Posting language
- English
- Posted
Similar jobs hiring now
Search current openingsOriginal job description
Archived listing. The details below describe a past opening.
We are seeking a highly skilled Senior/Lead Azure Cloud Data Engineer to design, develop, and optimize large-scale cloud-based data solutions on Microsoft Azure. The ideal candidate will have strong expertise in Azure Data Factory (ADF), Azure Databricks, PySpark, and Autosys, with hands-on experience building scalable data pipelines, data transformation frameworks, and enterprise-grade data platforms.
The candidate will collaborate with business stakeholders, architects, and development teams to deliver high-performance data solutions that support analytics, reporting, regulatory, and operational requirements.
Key Responsibilities
- Design, develop, and maintain scalable data ingestion and transformation pipelines using Azure Data Factory (ADF).
- Build and optimize data processing solutions using Azure Databricks and PySpark.
- Develop and maintain batch and near real-time data integration workflows.
- Implement data quality, validation, reconciliation, and monitoring frameworks.
- Schedule, monitor, and troubleshoot jobs using Autosys and cloud-native orchestration tools.
- Design and implement robust ETL/ELT solutions leveraging Azure cloud services.
- Optimize Spark jobs for performance, scalability, and cost efficiency.
- Collaborate with Data Architects and Solution Architects to define enterprise data models and standards.
- Support production deployments, incident management, and performance tuning activities.
- Ensure adherence to security, governance, and compliance standards.
- Mentor junior engineers and provide technical leadership for project delivery.
- Participate in code reviews, architecture discussions, and best practice implementation.
Required Qualifications
- Bachelor’s or master’s degree in computer science, Information Technology, Engineering, or related field.
- 10+ years of experience in Data Engineering and Data Warehousing.
- 5+ years of experience with Azure Data Platform technologies.
- Strong hands-on experience with:
- Azure Data Factory (ADF)
- Azure Databricks
- ADLS
- MYSQL
- PySpark
- SQL
- Autosys
- Experience designing and implementing ETL/ELT frameworks.
- Strong understanding of distributed computing and Spark architecture.
- Experience working with large-scale structured and semi-structured datasets.
- Proficiency in Python and SQL programming.
- Experience with source control tools such as Git/Azure DevOps.
More about this job
Responsibilities
Design, build, and optimize scalable Azure data ingestion, transformation, and integration solutions using Azure Data Factory, Databricks, PySpark, and related tools. Maintain production workflows, improve performance and data quality, support governance and compliance, and provide technical leadership to project teams.
Requirements
Requires 10+ years of data engineering and data warehousing experience, including 5+ years with Azure data platform technologies and hands-on expertise in Azure Data Factory, Databricks, ADLS, MySQL, PySpark, SQL, and Autosys. Candidates need a relevant bachelor's or master's degree, strong Python and SQL skills, experience with ETL/ELT frameworks and large datasets, and familiarity with Git or Azure DevOps.
Skills
- Azure Data Factory
- Azure Databricks
- PySpark
- Autosys
- Azure Data Lake Storage
- MySQL
- SQL
- Python
- ETL/ELT
- Data Pipeline Development
- Spark Architecture
- Data Quality
- Git
- Azure DevOps
- Data Warehousing
- Cloud Data Engineering
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Engineering
- Management & Leadership
Keywords
- Microsoft Azure
- Azure Data Factory
- Azure Databricks
- Azure Data Lake Storage
- PySpark
- Spark
- Autosys
- MySQL
- SQL
- Python
- ETL/ELT
- Data Engineering
- Data Warehousing
- Data Pipelines
- Data Ingestion
- Data Transformation
- Batch Processing
- Near Real-Time Integration
- Data Quality
- Data Validation
- Data Reconciliation
- Data Monitoring
- Distributed Computing
- Structured Data
- Semi-Structured Data
- Git
- Azure DevOps
- Data Governance
- Regulatory Compliance
- Performance Optimization