PySpark Data Engineer

Python / PySpark Data Engineer

Job Type: Contract (1 year)

Location: Montréal, QC – Hybrid (3 days onsite)

Hourly Rate: $55-65/hour

Your New Company

You will join a leading global financial services organisation that is investing in a strategic Client Data Confidentiality and data-usage monitoring platform. This platform supports the monitoring, protection and reporting of sensitive client data across multiple business areas.

Your New Role

As a Python / PySpark Data Engineer, you will develop and enhance scalable data-processing capabilities used for critical client and regulatory reporting. Your focus will be on building reliable data pipelines, integrating new data sources and applications, and improving the quality, completeness and performance of data across the monitoring platform.

You will work closely with developers in Montréal and stakeholders in New York to deliver business-critical platform enhancements.

What You Need to Do

• Design and develop scalable data-processing pipelines using Python, PySpark and Databricks.
• Build complex SQL and data-processing solutions on Snowflake.
• Develop ingestion and transformation capabilities for multiple data sources, including relational and NoSQL databases.
• Integrate data from MongoDB, KDB, and application or user-interface activity sources.
• Onboard new databases and applications into the monitoring platform.
• Expand monitoring coverage across additional business areas and data environments.
• Implement data-quality, validation and reconciliation controls.
• Ensure the completeness and accuracy of data used in client and regulatory enquiries.
• Troubleshoot data-processing issues and resolve data discrepancies.
• Optimise pipelines for performance, scalability, reliability and maintainability.
• Collaborate with geographically distributed technical and business stakeholders.

What You’ll Need to Succeed

• 5 years of hands-on software engineering or data engineering experience.
• Strong professional development experience with Python.
• Hands-on experience building distributed data-processing solutions with PySpark.
• Experience developing and maintaining data pipelines using Databricks.
• Advanced SQL development skills.
• Experience working with Snowflake or a comparable cloud-based data platform.
• Experience integrating data from heterogeneous database technologies and application sources.
• Strong understanding of data ingestion, transformation, validation and reconciliation.
• Demonstrated ability to troubleshoot complex data-processing and pipeline issues.
• Strong analytical and problem-solving capabilities.
• Ability to collaborate effectively with developers, business stakeholders and distributed teams.
• Exposure to MongoDB and KDB is considered an asset.
• Experience within financial services, data confidentiality, regulatory reporting or data-usage monitoring would be an advantage.

What You’ll Get in Return

You will have the opportunity to work on a highly visible data platform supporting sensitive client information and regulatory requirements within a global financial services environment. You will collaborate with experienced technology professionals, work with a modern data stack that includes Python, PySpark, Databricks, Snowflake and multiple database technologies, and contribute to a platform with significant business impact.

What You Need to Do Now

If you’re interested in this role, click “Apply now” to forward an up-to-date copy of your CV.

If this position is not quite right for you, but you are exploring new opportunities, please contact us for a confidential discussion about your career.

This posting is for an existing vacancy with the organisation.

AI may be used to screen, assess or select applicants for the position.