Closed listing
Data Engineer
- Toronto, ON, Canada
$45 to $55 CAD / hour · Contract
This opening has closed.
The original listing is kept below for reference.
Job details
- Pay
- $45 to $55 CAD / hour
- Job type
- Contract
- Closing date
- Posting language
- English
- Posted
Similar jobs hiring now
Search current openingsOriginal job description
Archived listing. The details below describe a past opening.
Role: Data Engineer
Hybrid: 3 days a week in office
Location: Toronto
Job Summary
An experienced Data Engineer with strong expertise in Big Data technologies to design, develop, and support enterprise-scale data platforms. The ideal candidate should possess hands-on experience in PySpark, Apache Spark, Kafka, Hadoop ecosystem components, and Apache NiFi, with a strong understanding of data ingestion, transformation, and real-time processing frameworks.
Key Responsibilities
- Design, develop, and optimize scalable data pipelines using PySpark, Spark, Hadoop, and Apache NiFi.
- Build and maintain batch and real-time data processing solutions.
- Develop and support Kafka-based streaming applications and event-driven architectures.
- Create and optimize ETL/ELT workflows for large-scale structured and unstructured datasets.
- Develop complex SQL queries for data extraction, transformation, validation, and troubleshooting.
- Implement data ingestion solutions from databases, APIs, files, and streaming sources.
- Monitor, troubleshoot, and enhance the performance of Spark jobs and data pipelines.
- Collaborate with architects, business analysts, and development teams to deliver high-quality data solutions.
- Support platform upgrades, deployments, testing, certification, and production releases.
- Ensure data quality, governance, security, and operational excellence across data platforms.
Mandatory Skills
- PySpark
- Apache Spark (Spark SQL, DataFrames)
- Apache Kafka
- Hadoop Ecosystem (HDFS, Hive, YARN)
- Apache NiFi
- SQL
- Python
Preferred Skills
- Spark Streaming
- Airflow / Oozie
- Hive
- Scala
- Jenkins, Bitbucket, Git
- JIRA, Confluence
- Cloud Platforms (GCP/AWS/Azure)
- Data Warehousing concepts and Dimensional Modeling
More about this job
Responsibilities
The Data Engineer will design, develop, and optimize scalable data pipelines and real-time processing solutions using Big Data technologies. They will also collaborate with cross-functional teams to ensure data quality, governance, and security across enterprise-scale platforms.
Requirements
Candidates must have strong expertise in PySpark, Apache Spark, Kafka, and the Hadoop ecosystem. Proficiency in SQL, Python, and experience with ETL/ELT workflows and data ingestion are mandatory for this role.
Skills
- Pyspark
- Apache Spark
- Kafka
- Hadoop
- Apache Nifi
- Sql
- Python
- Spark Streaming
- Airflow
- Oozie
- Hive
- Scala
- Jenkins
- Bitbucket
- Git
- Data Warehousing
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Engineering
- Finance & Accounting
Keywords
- Data Engineer
- Big Data
- Pyspark
- Apache Spark
- Kafka
- Hadoop
- Apache Nifi
- Etl
- Elt
- Sql
- Python
- Streaming
- Airflow
- Oozie
- Hive
- Scala
- Jenkins
- Bitbucket
- Git
- Data Warehousing
- Dimensional Modeling
- Gcp
- Aws
- Azure
- Data Pipelines
- Event-driven Architecture
- Data Governance
- Financial Services