Find work in Canada.

329,777 jobs across our Canadian network. How we count

Jobs in Canada

25+ jobs

Save this search

Added 1 min ago
Machine Learning Engineer
Dasro Consulting Inc.
Calgary, AB
Contract
Working experience with general Python data packages and building solutions in AWS
Added 33 mins ago
Engineering Co-op Student - January 2027
Hudbay Minerals
Princeton, BC
$65,624 to $68,744 CAD / year
Advancing Your Global Career: Elevate Your Future with Hudbay
Added 33 mins ago
Lead Test Engineer
Genesys
Toronto, ON · ON · AB · BC
Full-time
Strong proficiency in Java or Python, with experience contributing to test automation frameworks
Added 47 mins ago
Senior Python Engineer (AI skills)
Luxoft
Toronto, ON
Contract
…This role is suited for an experienced, self-directed software engineer who can operate with a high degree of autonomy, take ownership of outcomes, solve ambiguous problems, and deliver…
Added 47 mins ago
Data Engineer
SGS Consulting
Canada
Contract
Proficient in Python and SQL, with experience in PostgreSQL
Added 47 mins ago
Principal Engineer, Software Engineering / AI Engineer
Quantum Technology Recruiting Inc. (QTR)
Toronto, ON
$285,000 to $325,000 USD / year
Full-time
- Experience with Java and/or Python; React is highly valuable
Added 1 hour ago
AI/ML Platform Engineer (Cloud Infrastructure)
K&K Global Talent Solutions INC.
ON · Ontario, Canada
Full-time
Programming experience with Python or Go for automation
Added 1 hour ago
Software Engineer II
Socure
Toronto, ON · Greater Toronto Area, Canada
Full-time
Proficiency in at least one modern backend programming language and ecosystem, such as Java, Go, Python, or similar, with comfort working with cloud-native infrastructure, APIs, and data…
Added 1 hour ago
Senior Data Engineer
MatchBox Consulting Group
Vancouver, BC
Contract
MatchBox Consulting Group is currently seeking a Senior Data Engineer for a 6 month contract role with a client of ours located in Vancouver.
Added 1 hour ago
Big Data Engineer
Themesoft Inc.
Toronto, ON
Contract
Work with Parquet, ORC, Avro file formats on HDFS
Added 1 hour ago
Security Research Engineer – AI/ML
BULLTECH SOLUTIONS LTD
Toronto, ON
Full-time
- Strong hands-on software development skills, particularly Python, Java, Node, Go, C/C++ with experience in additional modern programming languages.
Added 1 hour ago
Senior Data Engineer
RAPS Consulting Inc
Mississauga, ON
$110,000 to $120,000 CAD / year
Full-time
We are looking for a Senior Data Engineer to join the Data Sourcing and Analytics team and support the design, development, and enhancement of enterprise data solutions across modern cloud…
Added 1 hour ago
Platform Engineer - AI/ML Infrastructure
K&K Global Talent Solutions INC.
Toronto, ON
$110,000 to $120,000 CAD / year
Full-time
Position: Platform Engineer - AI/ML Infrastructure
Added 1 hour ago
Software Engineer
ResMed
Halifax, NS · CA Canada (ResMed Halifax)
Full-time
At Resmed, technology has a direct impact on people's lives. As a Software Engineer within our Digital Product Engineering team, you'll help build and evolve the cloud-based platforms and…
Added 1 hour ago
Security Researcher Engineer
Zodiac Solutions, Inc
Toronto, ON
Full-time
- Strong hands-on software development skills, particularly Python, Java, Node, Go, C/C++ with experience in additional modern programming languages.
Added 1 hour ago
Senior Software Engineer, Machine Learning
Chorus
Toronto, ON
Full-time
Reports to: Technical CEO, working directly with our Founding Engineer and Founding Data Scientist
Added 1 hour ago
Senior Data Engineer
Ipsos
Canada
We’re looking for someone with a deep interest and expertise in data engineering and the dedication to apply that passion. We need someone with the technical expertise to expand and…
Added 1 hour ago
Senior Data Engineer
Extendicare
Markham, ON · Extendicare (Canada) Inc (Steeles Ave.)
Full-time
Advanced experience with Microsoft SQLrver, SSIS, Python, and ETL development methodologies.
Added 1 hour ago
Technology Analyst - Databricks Developer
Infosys
ON
Infosys is seeking an Technology Analyst - Azure Databricks - Data Engineer.
Added 2 hours ago
Senior DevSecOps Engineer
Tata Consultancy Services
Toronto, ON
$90,000 to $110,000 CAD / year
Full-time
…clients, investors, employees, and the communities it serves. With a highly skilled workforce supporting customers from coast to coast, TCS has consistently been recognized as a Top…
Added 2 hours ago
Principal Python Engineer (AI Experience REQUIRED)
Luxoft
Toronto, ON
Full-time
…This role is suited for an experienced, self-directed software engineer who can operate with a high degree of autonomy, take ownership of outcomes, solve ambiguous problems, and deliver…
Added 2 hours ago
Data Engineer
Tata Consultancy Services
Toronto, ON
$90,000 to $110,000 CAD / year
Full-time
Experience with Python or scripting for data transformation and automation.
Added 2 hours ago
Platform Engineer - AI/ML Infrastructure
Tata Consultancy Services
Toronto, ON
$90,000 to $110,000 CAD / year
Full-time
…clients, investors, employees, and the communities it serves. With a highly skilled workforce supporting customers from coast to coast, TCS has consistently been recognized as a Top…
Added 2 hours ago
Cloud Platform Engineer
EQ Bank
Toronto, ON
Full-time
The Cloud Platform Engineer is responsible for all aspects of our Kubernetes and Docker containers. He/she will be responsible for production support, administration, testing, and…
Added 2 hours ago
Cloud & DevOps Engineer Co-op (May 2027 | 12-16 months)
Vena Solutions
Remote, Canada
Internship
As a Cloud & DevOps Engineer Co-op within our SaaS Technology & Operations organization, you'll contribute to teams focused on Developer Experience, DevOps, and Site Reliability Engineering

Themesoft Inc.

Big Data Engineer

Toronto, ON, Canada

Job type: Contract
Apply by: Oct 22, 2026
Posting language: English
Posted: Sep 22, 2026

Apply About employer

Applications open in a new tab.

Original job description

Position: Big Data Engineer

Location: Toronto, On - Hybrid

Skills

Big Data & Spark
SQL
Unix/Shell Scripting

Key Responsibilities

Big Data & Spark

Design and develop Spark-Scala applications for large-scale data processing on Hadoop/CDP clusters
Build and optimize ETL/ELT pipelines using Spark DataFrames, Datasets and Spark SQL
Tune Spark jobs for performance (partitioning, caching, broadcast joins, shuffle optimization)
Migrate Spark 2 applications to Spark 3 on Cloudera CDP platforms
Work with Parquet, ORC, Avro file formats on HDFS

SQL & Data Engineering

Write complex HiveQL / Spark SQL queries including window functions, CTEs, subqueries and aggregations
Design and maintain Hive external/managed tables and partitioned datasets
Optimize slow-running queries and resolve correlated subquery issues
Work with HDFS encryption zones and data governance requirements

Unix / Shell Scripting

Develop and maintain bash shell scripts for job orchestration and automation
Handle error management, return codes, logging and alerting in shell scripts
Manage HDFS operations (hdfs dfs commands), file transfers, and data validation
Manage Kerberos authentication (kinit, keytab handling)

API Extraction & Integration

Build scripts and pipelines to extract data from REST APIs using curl and Python
Handle OAuth2 token generation, bearer token refresh and API health checks
Parse and process JSON API responses and load into HDFS/Hive
Manage pagination, error handling and retry logic for API calls
Work with enterprise API gateways and URL parameter construction

AI & Copilot Capabilities

Leverage GitHub Copilot / AI coding assistants to accelerate development
Use AI tools for code review, SQL generation, script debugging and documentation
Contribute to AI-assisted data quality and anomaly detection pipelines
Explore and implement LLM-based automation for repetitive data engineering tasks

Scheduling & Orchestration

Schedule and manage jobs using AAP (Ansible Automation Platform) / Control-M / cron
Build and maintain Ansible playbooks for automated deployments
Manage deployment pipelines including artifact versioning, Vault secret injection and environment-specific configuration
Monitor job health, handle failures and implement alerting

Nice to Have

Experience with Cloudera CDP (7.x) and migration from HDP
Knowledge of Kerberos, Vault, HDFS encryption zones
Familiarity with CI/CD pipelines (Helios, GitHub Actions)
Experience with MSSQL / JDBC connectivity from Spark
Understanding of AML / Financial regulatory data domains

Regards

Patrick Fernandez

Talent Acquisition Group - Strategic Recruitment Manager

Apply About employer

More about this job

Responsibilities

Design and develop Spark-Scala applications for large-scale data processing while building and optimizing ETL/ELT pipelines. Manage job orchestration, API integration, and data governance requirements within Hadoop/CDP clusters.

Requirements

Requires proficiency in Big Data technologies, Spark SQL, and Unix shell scripting for automation and data validation. Experience with Cloudera CDP, Kerberos authentication, and REST API integration is highly preferred.

Skills

Big Data
Spark
Scala
Hadoop
SQL
HiveQL
Unix
Shell Scripting
ETL
Python
HDFS
Ansible
Kerberos
REST APIs
Data Engineering

Visa sponsorship

Not detected in the job text

Hiring contact

Patrick Fernandez

Keywords

Big Data
Spark
Scala
Hadoop
CDP
ETL
ELT
Spark SQL
HiveQL
Unix
Shell Scripting
HDFS
Kerberos
REST APIs
Python
JSON
GitHub Copilot
Ansible
Control-M
CI/CD
Data Engineering
Data Governance
Parquet
Avro
ORC
OAuth2
API Gateway
Automation
Data Quality
Anomaly Detection

Report this job