Spark Developer (With Java)
- Mississauga, ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Experience
- Experience: 4 to 8 Years
- Posting language
- English
- Posted
Original job description
Job Title: Spark Developer (Java)
Experience: 4 to 8 Years
Location: Any
Employment Type: Full-time
Job Summary
We are seeking a highly skilled Spark Developer with strong Java expertise to design, develop, and optimize large-scale data processing applications. The ideal candidate will have hands-on experience with Apache Spark, Core Java, distributed computing, and big data technologies. The candidate will work closely with data engineers, architects, and business stakeholders to build scalable and high-performance data solutions.
Key Responsibilities
- Design, develop, and maintain scalable data processing applications using Apache Spark and Java.
- Develop batch and near real-time data pipelines for large datasets.
- Optimize Spark jobs for performance, memory management, and resource utilization.
- Write efficient and reusable code using Java 8/11 features.
- Work with distributed data processing frameworks and big data ecosystems.
- Collaborate with cross-functional teams to gather requirements and deliver data solutions.
- Troubleshoot production issues and provide performance tuning recommendations.
- Implement best practices for coding, testing, deployment, and monitoring.
- Participate in code reviews and technical design discussions.
- Ensure data quality, reliability, and scalability of applications.
Required Skills
- Strong programming experience in Core Java (Java 8 or above).
- Hands-on experience with Apache Spark (Spark Core, Spark SQL, DataFrames, Dataset APIs).
- Good understanding of distributed computing concepts.
- Experience with HDFS, Hive, Hadoop Ecosystem.
- Strong SQL skills and experience with relational databases.
- Experience with data ingestion and transformation frameworks.
- Familiarity with Linux/Unix environments.
- Knowledge of REST APIs and microservices architecture.
- Experience with Git and CI/CD processes.
Preferred Skills
- Experience with Spark Streaming, Kafka, Scala, or PySpark.
- Exposure to cloud platforms such as AWS, Azure, or GCP.
- Experience with Airflow, Oozie, or similar workflow orchestration tools.
- Understanding of Delta Lake, Iceberg, or Data Lake architectures.
- Knowledge of containerization tools such as Docker and Kubernetes.
Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
- 4+ years of software development experience.
- 2+ years of hands-on experience developing applications using Apache Spark and Java.
More about this job
Responsibilities
Design, develop, and maintain scalable Spark and Java applications and batch or near-real-time data pipelines. Optimize performance and resource use, troubleshoot production issues, and collaborate with teams to deliver reliable, high-quality data solutions.
Requirements
Requires strong Core Java skills, hands-on Apache Spark experience, knowledge of distributed computing and the Hadoop ecosystem, and strong SQL skills. Applicants need 4 or more years of software development experience, including at least 2 years developing applications with Spark and Java; a bachelor's or master's degree in a related field is listed.
Skills
- Core Java
- Apache Spark
- Spark Core
- Spark SQL
- DataFrames
- Dataset APIs
- Distributed Computing
- HDFS
- Hive
- Hadoop
- SQL
- Data Ingestion
- Data Transformation
- Linux/Unix
- REST APIs
- Git
Visa sponsorship
Not detected in the job text
Categories
- Software
- Technology
- Data & Analytics
Keywords
- Apache Spark
- Java 8
- Java 11
- Spark Core
- Spark SQL
- DataFrames
- Dataset APIs
- Distributed Computing
- Big Data
- HDFS
- Hive
- Hadoop Ecosystem
- SQL
- Relational Databases
- Data Ingestion
- Data Transformation
- Batch Processing
- Real-Time Data Pipelines
- Spark Streaming
- Kafka
- Scala
- PySpark
- AWS
- Azure
- GCP
- Airflow
- Oozie
- Delta Lake
- Iceberg
- Docker