Specialist - Data Engineering
- ON, Canada
- Ontario, Canada, Canada
$79,851 to $113,900 CAD / year · Full-time
Applications open in a new tab.
Job details
- Pay
- $79,851 to $113,900 CAD / year
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
Role Description
Job role : Spark and scala developer
Location : Mississauga ,Ontario
Job Summary
Seeking a Specialist with 5 to 7 years of experience in Scala and Apache Spark to design develop and optimize scalable big data processing solutions within the Hadoop Spark SparkSQL ecosystem
Job Description
Design develop and optimize scalable data processing applications using Scala and Apache Spark Collaborate with crossfunctional teams including data scientists analysts and architects to build and maintain robust data pipelines Analyze large and complex datasets to extract actionable insights supporting business objectives Develop highperformance applications in distributed computing environments leveraging SparkSQL Ensure code quality maintainability and adherence to Scala programming best practices Continuously evaluate and integrate emerging big data technologies and tools to enhance platform capabilities Troubleshoot and resolve complex data processing issues to maintain system reliability Document technical designs and implementation details for effective knowledge sharing
Roles and Responsibilities
Lead the development and deployment of big data solutions leveraging Scala Apache Spark and SparkSQL
Collaborate with stakeholders to gather requirements and deliver effective data solutions
Optimize Spark jobs for performance and scalability in production environments
Mentor junior team members and provide technical guidance on best practices
Conduct code reviews to ensure adherence to coding standards and promote continuous improvement
Monitor system performance proactively and address bottlenecks or failures
Participate in sprint planning estimation and agile ceremonies to deliver highquality software
Ensure compliance with data governance and security policies
Job role : Spark and scala developer
Location : Mississauga ,Ontario
Job Summary
Seeking a Specialist with 5 to 7 years of experience in Scala and Apache Spark to design develop and optimize scalable big data processing solutions within the Hadoop Spark SparkSQL ecosystem
Job Description
Design develop and optimize scalable data processing applications using Scala and Apache Spark Collaborate with crossfunctional teams including data scientists analysts and architects to build and maintain robust data pipelines Analyze large and complex datasets to extract actionable insights supporting business objectives Develop highperformance applications in distributed computing environments leveraging SparkSQL Ensure code quality maintainability and adherence to Scala programming best practices Continuously evaluate and integrate emerging big data technologies and tools to enhance platform capabilities Troubleshoot and resolve complex data processing issues to maintain system reliability Document technical designs and implementation details for effective knowledge sharing
Roles and Responsibilities
Lead the development and deployment of big data solutions leveraging Scala Apache Spark and SparkSQL
Collaborate with stakeholders to gather requirements and deliver effective data solutions
Optimize Spark jobs for performance and scalability in production environments
Mentor junior team members and provide technical guidance on best practices
Conduct code reviews to ensure adherence to coding standards and promote continuous improvement
Monitor system performance proactively and address bottlenecks or failures
Participate in sprint planning estimation and agile ceremonies to deliver highquality software
Ensure compliance with data governance and security policies
More about this job
Responsibilities
Lead the design, development, deployment, and optimization of scalable big data solutions using Scala, Apache Spark, and Spark SQL. Collaborate with stakeholders and cross-functional teams, guide junior developers, review code, monitor production systems, and ensure data governance and security compliance.
Requirements
The role seeks 5 to 7 years of experience with Scala and Apache Spark in big data processing environments. Candidates should be able to build and optimize distributed data applications, maintain code quality, troubleshoot complex issues, and provide technical guidance.
Skills
- Scala
- Apache Spark
- Spark SQL
- Big Data Processing
- Data Pipeline Development
- Distributed Computing
- Performance Optimization
- Data Analysis
- Troubleshooting
- Code Review
- Technical Mentoring
- Data Governance
- Data Security
- Agile Methodologies
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
- Engineering
Keywords
- Scala
- Apache Spark
- Spark SQL
- Hadoop
- Big Data
- Data Engineering
- Data Processing
- Data Pipelines
- Distributed Computing
- Performance Optimization
- Data Analysis
- Production Environments
- Code Quality
- Code Reviews
- Technical Mentoring
- Data Governance
- Data Security
- Agile
- Sprint Planning
- Cross-Functional Collaboration