RQ11673 - Junior Report Developer

MUST HAVE Qualifications:

• Degree or diploma in Computer Science, Software Engineering, Information Technology, or a related field.

• Experience developing data solutions using Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Databricks, Delta Lake and Lakehouse architecture principles.

• Experience working with Azure DevOps, Git, CI/CD pipelines, and automated deployment practices.

• Strong proficiency in Python, SQL, and Spark-based data processing frameworks.

• Knowledge of data governance, security, privacy, and access control principles.

• Experience with REST APIs, application integration patterns, and automation frameworks.

Key Responsibilities

Data Engineering & Pipeline Development

• Design, develop, and maintain scalable batch and near real-time data pipelines using Azure Data Factory (ADF), Databricks, and related cloud technologies.

• Develop data ingestion frameworks to onboard data from various source systems into the Ontario Data Platform.

• Build and optimize ETL/ELT processes to transform, cleanse, and prepare data for analytics and reporting.

• Implement data validation, monitoring, and reconciliation processes to ensure data quality and reliability.

Databricks Development

• Develop and maintain notebooks, workflows, and data processing jobs using Databricks.

• Build data transformation pipelines leveraging Spark and Databricks Lakehouse architecture.

• Support the implementation of medallion architecture patterns (Bronze, Silver, Gold).

• Monitor and troubleshoot Databricks jobs, workflows, and performance issues.

• Develop and manage Delta Lake tables and optimize data processing workloads for performance and scalability.

Data Storage & Integration

• Design and manage data solutions utilizing Azure Data Lake Storage (ADLS).

• Collaborate with source system teams to establish secure and efficient data integration patterns.

• Implement data partitioning, optimization, and archival strategies to improve performance and scalability.

• Support metadata management, data cataloguing, and governance initiatives.

Software Development & Automation

• Develop reusable frameworks, scripts, and utilities to automate data engineering processes.

• Participate in DevOps practices, including CI/CD pipelines, code reviews, testing, and deployment automation.

• Maintain technical documentation, design specifications, and operational procedures.

• Contribute to platform enhancements and continuous improvement initiatives.

• Apply software engineering best practices including version control, test automation, code quality standards, and secure coding principles.

Collaboration & Stakeholder Engagement

• Work closely with architects, product managers, data scientists, analysts, and business stakeholders to understand requirements and deliver technical solutions.

• Participate in Agile teams and contribute to sprint planning, estimation, and delivery activities.

• Provide technical guidance and knowledge sharing to team members and partners.

• Participate in solution architecture reviews and contribute to technology roadmaps and design decisions.