Pyspark ETL Developer
- Toronto, ON, Canada
- Greater Toronto Area, Canada, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
Duration:
5 Month contract
Must have:
- Python and PySpark programing languages to manipulate data and draw insights from large datasets
- Experience with AWS EMR, AWS Cloud
- Hands-on programming skills, able to write modular, maintainable code, preferably Python & SQL. Good understanding of SQL, dimensional modeling, and analytical big data warehouses
- Experience with Apache Airflow
- Experience with Python, PySpark programming in the context of data processing / data science applications
- Profound knowledge and experience in shell scripting in Linux/Unix environment. Experience in creating and maintaining documentation of project ETL design, technical artifacts and ETL environment
A government health data organization.
Responsibilities:
Understand systems and Business processes related to the products in their assigned portfolio
Works closely with client analytical teams to develop, maintain and support products using Python and/or R and PySpark programming languages.
Provides support and guidance on creating top-quality clean code and participates in code review.
Analyses Software deficiencies/problems and support to fix them
Prepares documentation, reports, and presentations related to projects and activities.
More about this job
Responsibilities
Develop, maintain, and support data products using Python, R, and PySpark in collaboration with client analytical teams. Analyze and resolve software issues, contribute to code reviews and clean-code practices, and prepare project documentation, reports, and presentations.
Requirements
Requires strong Python and PySpark programming skills, experience with AWS EMR and AWS Cloud, and proficiency in SQL, dimensional modeling, analytical data warehouses, and Apache Airflow. Candidates should also have Linux/Unix shell scripting experience and be able to write maintainable code and document ETL designs and technical artifacts.
Skills
- Python
- PySpark
- AWS EMR
- AWS Cloud
- SQL
- Dimensional Modeling
- Analytical Data Warehousing
- Apache Airflow
- Shell Scripting
- Linux/Unix
- ETL Design
- Data Processing
- Data Science
- Code Review
- Technical Documentation
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
- Healthcare
- Government & Public Sector
Keywords
- Python
- PySpark
- R
- AWS EMR
- AWS Cloud
- SQL
- Dimensional Modeling
- Analytical Data Warehouses
- Apache Airflow
- Shell Scripting
- Linux/Unix
- ETL
- Data Processing
- Data Science
- Big Data
- Data Analysis
- Modular Code
- Maintainable Code
- Code Review
- Software Troubleshooting
- Technical Documentation
- Government Health Data
- Data Products