Staff Data Platform Engineer (Streaming Platform)
- Toronto, ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
About the Company
TekRek is partnered with a high-growth product company building the data platform behind a modern telematics API, supporting real-time ingestion, storage, enrichment, and analytics for high-volume IoT and time-series workloads at petabyte scale.
The Role
This is not a traditional data engineering seat; you will own the long-term buildout of the data platform powering the API and analytics layer, drive architectural direction, and partner with product, engineering, and leadership to deliver scalable abstractions and reusable components.
What You’ll Do
- Own projects enhancing data replication, storage, enrichment, and reporting capabilities
- Build and optimize streaming and batch pipelines that support the core product and API
- Design scalable storage solutions for petabytes of IoT and time-series data
- Develop and maintain real-time ingestion systems to support growing data volumes
- Implement distributed tracing, data lineage, and observability to improve monitoring and troubleshooting
What You Bring (Must Have)
- 8 plus years of experience in platform engineering or data engineering
- 6 plus years designing and optimizing data pipelines at TB to PB scale
- Strong Java proficiency with a focus on clean, maintainable production code
- Strong system design skills for big data and real-time workflows
- Experience with lakehouse and streaming ecosystems such as Iceberg or Delta, plus Kafka or Flink or Spark
Tech Stack
- Languages: Java, Python
- Framework: Spring Boot
- Storage: AWS S3, DynamoDB, Apache Iceberg, Redis
- Streaming: AWS Kinesis, Kafka, Flink
- ETL: AWS Glue, Spark
- Serverless: SQS, EventBridge, Lambda, Step Functions
- Infrastructure as Code: AWS CDK
- CI/CD: GitHub Actions
More about this job
Responsibilities
Own the development of platform capabilities for data replication, storage, enrichment, reporting, and scalable streaming and batch pipelines. Design petabyte-scale storage and real-time ingestion systems, and implement data lineage, distributed tracing, and observability.
Requirements
Requires 8+ years of platform or data engineering experience and 6+ years designing and optimizing data pipelines at terabyte-to-petabyte scale. Candidates should have strong Java skills, big data and real-time system design expertise, and experience with lakehouse and streaming technologies such as Iceberg or Delta and Kafka, Flink, or Spark.
Skills
- Java
- Python
- Data Platform Engineering
- Data Engineering
- Streaming Pipelines
- Batch Pipelines
- Big Data System Design
- Real-Time Data Ingestion
- Data Replication
- Lakehouse Architecture
- Apache Iceberg
- Delta Lake
- Kafka
- Flink
- Spark
- Distributed Tracing
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Software
- Data & Analytics
- Engineering
- Transportation
Keywords
- Java
- Python
- Spring Boot
- AWS S3
- DynamoDB
- Apache Iceberg
- Delta Lake
- Redis
- AWS Kinesis
- Kafka
- Flink
- Spark
- AWS Glue
- SQS
- EventBridge
- AWS Lambda
- Step Functions
- AWS CDK
- GitHub Actions
- Data Platform Engineering
- Data Pipelines
- Streaming
- Batch Processing
- Real-Time Ingestion
- Petabyte-Scale Storage
- IoT Data
- Time-Series Data
- Data Lineage
- Distributed Tracing
- Observability