Python / PySpark Data Engineer
Pay Rate: $60/hr to $65/hr on Inc
Your New Company
Your New Role
We are seeking a Python / PySpark Data Engineer with 3–5 years of experience to develop and enhance a strategic Client Data Confidentiality and data-usage monitoring platform. The role will focus on building scalable data pipelines, onboarding new data sources, and ensuring the quality and reliability of data used for critical client and regulatory reporting.
What You Need to Do
What You'll Need to Succeed:
Key Responsibilities
The successful candidate will design and develop scalable data-processing pipelines using Python, PySpark and Databricks.
You will develop complex SQL and data-processing solutions on Snowflake.
The role requires building ingestion and transformation capabilities for multiple data sources, including MongoDB, KDB and application/UI activity data.
You will onboard new databases and applications into the monitoring platform and expand coverage across business areas.
The position involves implementing data quality, validation, and reconciliation controls to ensure completeness and accuracy of data used for client and regulatory inquiries.
You will troubleshoot data-processing issues and optimize pipelines for performance, scalability, and reliability.
The successful candidate will collaborate with Montreal-based developers and New York stakeholders to deliver business-critical enhancements.
Required Skills
- 3–5 years of hands-on software or data engineering experience.
- Strong development experience in Python.
- Hands-on experience with PySpark and experience developing data pipelines using Databricks.
- Strong SQL development skills, with experience with Snowflake or comparable database platforms.
- Experience integrating data from heterogeneous database technologies. Exposure to MongoDB and KDB is a plus.
- Strong analytical, debugging, and problem-solving skills.
What You'll Get in Return:
You will have the opportunity to work with a talented team of professionals in a dynamic and supportive environment. The organization is offering competitive compensation, professional development opportunities, and the chance to contribute to a strategic data platform supporting critical client and regulatory reporting initiatives.
What You Need to Do Now:
If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV, or call us now.
If this job isn't quite right for you, but you are looking for a new position, please contact us for a confidential discussion on your career.
This posting is for an existing vacancy with the organization.
AI may be used to screen, assess or select applicants for the position.
