Data Engineer – PySpark & SQL
- Mississauga, ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
Job Title: Data Engineer – PySpark & SQL
Location: Mississauga, ON
Fulltime
We are looking for an experienced 3+ years of Data Engineer with strong hands-on expertise in PySpark, Python, SQL, and data engineering. The ideal candidate should have experience building scalable data pipelines and working with large datasets in cloud or distributed data environments.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark and Python.
- Develop complex and optimized SQL queries for data processing and transformation.
- Build ETL/ELT workflows to ingest, transform, and process large datasets.
- Perform data cleansing, validation, transformation, and performance optimization.
- Work with structured and semi-structured datasets.
- Troubleshoot data quality and pipeline performance issues.
- Collaborate with business, analytics, and engineering teams to understand data requirements.
- Follow data engineering best practices, coding standards, and documentation processes.
Required Skills
- Strong hands-on experience with PySpark / Apache Spark.
- Excellent proficiency in SQL, including complex queries, joins, CTEs, window functions, and query optimization.
- Good programming experience with Python.
- Experience developing ETL/ELT data pipelines.
- Good understanding of data warehousing and data modeling concepts.
- Experience working with large-scale/distributed data processing.
- Exposure to at least one major cloud platform such as AWS, Azure, or GCP.
- Strong problem-solving and debugging skills.
Certification Requirement
Candidate must hold at least one relevant, active Data Engineering or Cloud certification, preferably from:
- Databricks Certified Generative AI Engineer Associate
- Google Professional Machine Learning Engineer
- NVIDIA Certified Professional - Agentic AI (NCP-AAI)
- AWS Certified AI Practitioner
More about this job
Responsibilities
Design, develop, and maintain scalable data pipelines and ETL/ELT workflows using PySpark, Python, and SQL to process large structured and semi-structured datasets. Optimize queries and pipeline performance, address data quality issues, and collaborate with business, analytics, and engineering teams to meet data requirements.
Requirements
Requires at least 3 years of data engineering experience, strong hands-on expertise in PySpark or Apache Spark, Python, SQL, and ETL/ELT pipeline development, plus knowledge of data warehousing, data modeling, and distributed processing. Candidates must hold at least one relevant active data engineering or cloud certification; experience with a major cloud platform and strong problem-solving skills are also required.
Skills
- PySpark
- Apache Spark
- Python
- SQL
- ETL/ELT
- Data Pipeline Development
- Data Warehousing
- Data Modeling
- Distributed Data Processing
- Cloud Platforms
- Data Cleansing
- Data Validation
- Query Optimization
- Performance Optimization
- Troubleshooting
- Data Quality
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Engineering
Keywords
- Data Engineering
- PySpark
- Apache Spark
- Python
- SQL
- ETL
- ELT
- Data Pipelines
- Data Processing
- Large Datasets
- Distributed Computing
- Cloud Platforms
- AWS
- Azure
- GCP
- Data Warehousing
- Data Modeling
- Structured Data
- Semi-Structured Data
- Data Cleansing
- Data Validation
- Query Optimization
- Performance Optimization
- Data Quality
- Google Professional Machine Learning Engineer
- NVIDIA Certified Professional - Agentic AI
- AWS Certified AI Practitioner