Data Engineer
- Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Posting language
- English
- Posted
Original job description
Mandatory Skills:Databricks and data engineering, streaming data ingestion stream pipelines and Unity Catalog.ScalaAWS (s3, Kinesis and Lambda)TerraformOther skills:Working with LLMs, particularly Claude.Creating MCPs, Skills and agents.Agents swarm and harness.Data governance and protection: PII, PCI, data retention and lifecycle.Domain exposure would be a plus:SubscriptionsBilling, invoicing and payments.
More about this job
Responsibilities
Build and manage data engineering solutions, including streaming ingestion pipelines, using Databricks, Unity Catalog, Scala, AWS, and Terraform. Work with LLMs and agent-based systems while supporting data governance and protection requirements.
Requirements
The role requires experience with Databricks and data engineering, streaming pipelines, Unity Catalog, Scala, AWS services, and Terraform. Familiarity with LLMs, Claude, MCPs, agent swarms, and data protection practices is also relevant; subscription billing, invoicing, and payments domain exposure is a plus.
Skills
- Databricks
- Data Engineering
- Streaming Data Ingestion
- Stream Pipelines
- Unity Catalog
- Scala
- AWS
- Amazon S3
- Amazon Kinesis
- AWS Lambda
- Terraform
- Large Language Models
- Claude
- Model Context Protocol
- AI Agents
- Data Governance
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
Keywords
- Databricks
- Data Engineering
- Streaming Data Ingestion
- Stream Pipelines
- Unity Catalog
- Scala
- AWS
- Amazon S3
- Amazon Kinesis
- AWS Lambda
- Terraform
- Large Language Models
- Claude
- Model Context Protocol
- Skills
- AI Agents
- Agent Swarms
- Agent Harnesses
- Data Governance
- Data Protection
- Personally Identifiable Information
- Payment Card Industry Data
- Data Retention
- Data Lifecycle
- Subscriptions
- Billing
- Invoicing
- Payments