Senior Python Developer (PySpark & AWS Platform Engineering)

Position: Senior Python Developer (PySpark & AWS Platform Engineering)

Location: 4days/week Onsite- Toronto, ON

Job type: Long-term Contract

Note: The final/client interview will be conducted on-site.

We are seeking an elite Senior Python Developer with a mastery of PySpark and AWS distributed infrastructure to drive the engineering and integration of our core data platform ecosystem. This role is embedded within our Data Engineering infrastructure organization, focusing directly on our Analytical Work Bench (AWB) - an enterprise-grade Integrated Development Environment (IDE) used by our data science and research teams.

You will spearhead the integration and rationalization of AWB with internal applications, while simultaneously developing a custom enterprise IDE based on the JupyterLab environment. This is not a traditional data-analytics role; it requires a seasoned Python Software Engineer who builds scalable data infrastructure tools, applies strict software design patterns, and optimizes distributed compute workloads at scale.

Core Responsibilities

• Application Rationalization & Integration: Design and implement the software framework to rationalize, integrate, and streamline the Analytical Work Bench (AWB) platform with secondary internal applications.
• Custom IDE Engineering: Develop, extend, and package a customized enterprise engineering IDE based on the JupyterLab ecosystem using proprietary client frameworks and Python extensions.
• Big Data Engineering: Ingest and integrate massive datasets into the research environment by architecting highly reliable, parallel data structures using PySpark on AWS EMR.
• Software Quality & Automation: Enforce top-tier code health and execution patterns across the platform by implementing automated unit-testing pipelines via Pytest, static code analysis via Pylint, and build task runners like Gulp.
• Package Management & Deployment: Oversee the lifecycle and distribution of internal Python modules through PyPI, automating release pipelines to live staging and production environments using Jenkins and UrbanCode.
• Technical Collaboration: Partner directly with technical end-users and data science teams to distill highly complex business demands into performant platform architectures.

Required Skillsets

• Python Mastery & Seniority: 8+ years of production-level experience writing scalable Core Python applications and advanced PySpark processing pipelines.
• AWS Big Data Ecosystems: Deep, hands-on experience orchestrating and fine-tuning distributed clusters on AWS EMR and associated cloud storage/compute frameworks.
• Code Craftsmanship & Testing: A minimum of 2+ years of concrete experience utilizing Pytest for structural testing and Pylint for code style and quality enforcement.
• Build Automation & Packaging: 5+ years of experience managing code distribution artifacts through PyPI and automating asset/build runners using Gulp.
• Enterprise CI/CD Workflows: 2+ years of practical pipeline experience setting up and supporting continuous deployment structures via Jenkins and UrbanCode.
• Education: A University degree in Computer Science, Software Engineering, or an equivalent technical discipline is preferred.

Preferred Qualifications / Nice to Have

• Proven background customizing, scripting extensions for, or engineering modules inside JupyterLab.
• Frontend experience with React to develop user interfaces, control hooks, or dashboard integrations for developer workspaces.
• Cross-functional domain understanding of classic Data Engineering architectures and Data Science modeling methodologies.