Job Title: Spark Developer (Java)
Experience: 4 to 8 Years
Location: Any
Employment Type: Full-time
Job Summary
We are seeking a highly skilled Spark Developer with strong Java expertise to design, develop, and optimize large-scale data processing applications. The ideal candidate will have hands-on experience with Apache Spark, Core Java, distributed computing, and big data technologies. The candidate will work closely with data engineers, architects, and business stakeholders to build scalable and high-performance data solutions.
Key Responsibilities
- Design, develop, and maintain scalable data processing applications using Apache Spark and Java.
- Develop batch and near real-time data pipelines for large datasets.
- Optimize Spark jobs for performance, memory management, and resource utilization.
- Write efficient and reusable code using Java 8/11 features.
- Work with distributed data processing frameworks and big data ecosystems.
- Collaborate with cross-functional teams to gather requirements and deliver data solutions.
- Troubleshoot production issues and provide performance tuning recommendations.
- Implement best practices for coding, testing, deployment, and monitoring.
- Participate in code reviews and technical design discussions.
- Ensure data quality, reliability, and scalability of applications.
Required Skills
- Strong programming experience in Core Java (Java 8 or above).
- Hands-on experience with Apache Spark (Spark Core, Spark SQL, DataFrames, Dataset APIs).
- Good understanding of distributed computing concepts.
- Experience with HDFS, Hive, Hadoop Ecosystem.
- Strong SQL skills and experience with relational databases.
- Experience with data ingestion and transformation frameworks.
- Familiarity with Linux/Unix environments.
- Knowledge of REST APIs and microservices architecture.
- Experience with Git and CI/CD processes.
Preferred Skills
- Experience with Spark Streaming, Kafka, Scala, or PySpark.
- Exposure to cloud platforms such as AWS, Azure, or GCP.
- Experience with Airflow, Oozie, or similar workflow orchestration tools.
- Understanding of Delta Lake, Iceberg, or Data Lake architectures.
- Knowledge of containerization tools such as Docker and Kubernetes.
Qualifications
- Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
- 4+ years of software development experience.
- 2+ years of hands-on experience developing applications using Apache Spark and Java.
