Senior Data Engineer
- Stellarton, NS, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
Sobeys is full of exciting opportunities, and we are always looking for bright new talent to join our team! We currently have a full time opportunity for a Senior Data Engineer role. This role can be based out of our offices located in Stellarton; NS, Mississauga, ON
Senior Data Engineer role will be part of the Data Integration Hub (DIH) within the 'Integration and COEs' department of Sobeys.
- Part of the enhancement and maintenance of DIH to support major program deliverables.
- Work closely with business SMEs, architects and technical leads from various divisions within Sobeys.
- Highly experienced in translating data requirements to design and implementation.
- Highly skilled in Azure Databricks, specifically streaming and event driven solutions.
- Highly skilled in performance and cost optimization of solutions.
- Solid expertise in CI/CD practices.
- Provides mentorship to junior engineers and serve as trusted advisor to leadership team.
- Significant experience (5+) developing with Spark (python/Vscode/Databricks Notebooks preferably) and specifically within Azure Databricks.
- Strong knowledge on DLT framework and structured streaming.
- Experienced in implementing scalable data ingestion frameworks.
- Experienced in enabling and using Unity catalog.
- Experienced in building a resilient solution for pipelines supporting operational systems.
- Experienced in engaging with the team towards a common sprint goal and be collaborative.
- Experienced in establishing Key Design Decisions and communicating with architects and core team for alignment.
- Actively engage in performance and cost optimization of the solution.
- Ability to identify redundant manual work and share opportunities to enable automation.
- Azure Dev Ops - CI/CD.
- Proven experience in test driven development.
- Azure and Databricks certification is a plus.
Sobeys is one of Canada’s leading grocery retailers, with more than 1,600 stores across all 10 provinces and banners including Sobeys, Safeway, IGA, Foodland, FreshCo, Thrifty Foods, and Lawtons Drug Stores. Our 128,000 teammates and franchise affiliates are passionate about delivering great food and exceptional experiences to our customers and communities.
Learn More About Our Story And Culture
Who We Are | Why Work With Us
Total Rewards
We offer a Total Rewards package designed to support teammates at work and in life. Depending on role and eligibility, teammates may receive health and dental benefits, retirement and savings programs including an Employee Share Ownership Plan, a 10% in-store discount at participating banners, virtual healthcare and an Employee and Family Assistance Program, learning and development opportunities, parental leave top-up, and paid vacation.
Sobeys is committed to providing a compensation structure that is flexible, equitable and competitive in the market to enable performance and growth. To learn more about this opportunity including the expected range of compensation in accordance with Pay Transparency Legislation where required please click the “I’m interested” or "Apply" button above. Individual compensation is determined based on qualifications, experience, and internal equity within the range provided.
Additional Information
External websites may share our organization's job postings which includes compensation information based on similar roles and market benchmarks. These figures are provided for general comparison purposes only and are not issued or verified by our organization.
We may use Artificial Intelligence (AI) tools to support efficiencies in the candidate screening, assessment, and recruitment processes. These AI tools do not make hiring decisions on behalf of the Company. Hiring decisions are made by our Hiring Teams.
Sobeys is committed to creating accessible and inclusive hiring processes. We will work with applicants requesting accommodation at any stage of the recruitment process.
Please note: Successful candidates will be required to provide documentation to prove their legal ability to work in the position during the onboarding process. Documentation will be assessed by the employer prior to commencement of work.
More about this job
Responsibilities
Enhance and maintain the Data Integration Hub, translating business data requirements into designs and implementations for scalable, resilient data pipelines, including streaming and event-driven solutions. Collaborate with business SMEs, architects, and technical leads; optimize performance and cost; establish design decisions; and mentor junior engineers.
Requirements
Requires 5+ years of Spark development experience, preferably using Python, VS Code, and Databricks Notebooks, with strong Azure Databricks expertise. Candidates should have experience with Delta Live Tables, structured streaming, scalable ingestion, Unity Catalog, resilient pipelines, Azure DevOps CI/CD, test-driven development, optimization, and collaborative technical design; Azure or Databricks certification is a plus.
Skills
- Apache Spark
- Python
- Azure Databricks
- Delta Live Tables
- Structured Streaming
- Data Ingestion Frameworks
- Unity Catalog
- Azure DevOps
- CI/CD
- Test-Driven Development
- Performance Optimization
- Cost Optimization
- Data Pipeline Resilience
- Data Integration
- Technical Mentorship
Benefits
- Health and Dental Benefits
- Retirement and Savings Programs
- Employee Share Ownership Plan
- Employee Discount
- Virtual Healthcare
- Employee and Family Assistance Program
- Learning and Development Opportunities
- Parental Leave Top-Up
- Paid Vacation
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Retail
- Management & Leadership
Keywords
- Apache Spark
- Python
- VS Code
- Databricks Notebooks
- Azure Databricks
- Delta Live Tables
- Structured Streaming
- Event-Driven Solutions
- Data Ingestion Frameworks
- Unity Catalog
- Azure DevOps
- CI/CD
- Test-Driven Development
- Performance Optimization
- Cost Optimization
- Data Pipelines
- Data Integration Hub
- Scalable Data Solutions
- Pipeline Resilience
- Key Design Decisions
- Sprint Goals
- Automation
- Azure Certification
- Databricks Certification
- Retail