ETL & Data Ingestion Engineer

Hiring an Azure Data Ingestion Engineer (Informatica Intelligent Data Management Cloud (IDMC) Data pipelines, MS Azure, Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure data pipelines, Kafka, Informatica, ETL, Data Warehouse, Python, and SQL). Preference will be given to candidates whose resumes clearly meet the required experience and requirements outlined below. Professionals with relevant experience may send their updated resume to Sriram.Parasa@brainhunter.com

Requirement Summary

• Job-Role/Title: Data Ingestion Engineer
• Anticipated Start Date: 3-4 weeks from offer (pending background checks)
• Job Type: Contract Opportunity. Initial contract for a 3-month term, with good possibilities of further extensions.
• Job Location: Toronto, ON / London, ON
• Work Style: Hybrid Work - 3 days per week in the office required (Tuesday and Thursday are mandatory).
• Work Hours: 37.5 hours/week, typically Monday-Friday, 9:00 am to 5:00 pm
• Contractor to work extra time or be on-call: Yes, on-call rotation (Monday to Sunday) might be required on a need basis.
• Rate Cap: Up to CA$ 85/Hr. (on Incorp), depending on relevant experience. The final compensation will be negotiated between the contractor and recruiter based on experience and engagement terms.
• Interview: Teams interviews; 2 interviews

NOTE:

• Successful personnel must have an active Reliability Status Security Clearance / be willing to undergo the Security Clearance process
• Eligibility Criteria: The consultant must be a resident of Canada for at least 5+ years to be eligible to apply for the client's Reliability Status Security Clearance.
• References Required (Yes/No): YES, must share at least 2 professional references from recent projects.

Required Experience, Skills, and Qualifications:

• Informatica Intelligent Data Management Cloud (IDMC) Data pipelines, MS Azure, Azure Databricks, Azure Data Factory (ADF), Azure Data Lake Storage (ADLS), Azure data pipelines, Python
• 5+ years of prior experience in a data operations or development role (e-g ETL, Data Warehouse) with 2+ years of recent development experience in IDMC, MS Azure, and Databricks
• Demonstrated experience in designing and implementing data ingestion systems using MS-Azure /Databricks
• Experience in Kafka, CI/CD, and CDC tools preferred
• Solid programming experience with Python or SQL
• Experience in setting up monitoring and controls for MS Azure data pipelines (prior experience with Data warehousing/ other data technologies is accepted)
• Strong knowledge in service management, including incident and change management procedures.
• Problem-solving skills and the ability to troubleshoot complex issues.
• Contributing to positive team culture and promoting common goals, team success, and business outcomes
• Valuing and appreciating different views and bringing in a collaborative approach to achieving consensus.
• Encourages inclusivity and a sense of unity within all operational areas of the Canada Data team.

___________________________________________________________

Position Overview: As a Data Ingestion Engineer, you will join a dynamic and high-performing Canada Life Data team dedicated to providing Data and AI solutions by ensuring the robustness, scalability, and reliability of data ingestion systems within our organization. Your key focus will be on designing, implementing, and maintaining the pipelines necessary for efficient and seamless data ingestion from various sources into our data platforms. You will partner with cross-functional teams, enterprise, and portfolio architects to enhance operational performance, troubleshoot issues, and implement best practices for data reliability.

Key Responsibilities & Deliverables:

• Design, develop, test, and support end-to-end ingestion pipelines using Informatica IDMC in the MS Azure data platform that supports the consumption of structured and unstructured data from various sources with the use of Azure Data Factory and Databricks
• Collaborate with cross-functional teams to understand data requirements, design efficient data ingestion patterns and workflows.
• Implement and maintain controls and alerts to monitor data pipelines to proactively identify data-related issues. Ensure data quality and integrity are maintained across the pipelines.
• Collaborate with support teams to triage incidents to quickly resolve major incidents, system outages, or disruptions.
• Automate processes to support smooth operation of data pipelines to improve reliability, efficiency, and scalability using MS Azure tools.
• Conduct performance and capacity assessments, identify bottlenecks, and develop plans to improve performance and reduce operational costs.
• Focus on implementing data governance and compliance controls and security measures to protect sensitive data.
• Apply best practices for creation and maintenance of data pipelines, including scalability, reusability, and reliability.
• Document system configurations, processes, and troubleshooting procedures.

___________________________________________________________

How to Apply: Please email your up-to-date Resume/CV to Sriram.Parasa@brainhunter.com

Thank you for your interest in working with us. We'll contact only those candidates who are shortlisted for the next steps in the hiring process.

Thank you, and have a wonderful day!