At a glance
Automatically prepared from the listing. Check the original description for the full requirements.
Jump to the original descriptionResponsibilities
Design and implement ETL jobs using Python and Spark while migrating legacy SSIS processes to modern platforms. Maintain data pipelines and collaborate with cross-functional teams to translate business requirements into technical specifications.
Requirements
Requires strong proficiency in Python or Apache Spark for data processing and ETL development. Candidates must have advanced SQL knowledge for writing complex queries and optimizing database performance.
Working hours
40 hours per week
Skills
- Python
- SQL
- Apache Spark
- ETL Development
- SSIS
- SQL Server
- Data Pipeline Optimization
- Data Integrity
- Agile Methodology
- Database Performance Tuning
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Engineering
- Consulting
Keywords
- Python
- SQL
- Apache Spark
- ETL
- SSIS
- SQL Server
- Data Engineering
- Data Pipelines
- Agile
- Sprint Planning
- Stored Procedures
- Database Optimization
- Data Quality
- Data Architecture
- IT Consulting
Original job description
Key Responsibilities
- Design, develop, and implement new ETL (Extract, Transform, Load) jobs using Python and/or Spark to support various data initiatives.
- Migrate existing ETL processes from SSIS and SQL Server to modern data platforms, ensuring data integrity and performance.
- Maintain and optimize existing data pipelines and ETL processes for efficiency, reliability, and scalability.
- Develop and implement robust testing strategies for all ETL jobs to ensure data quality and accuracy.
- Collaborate with data architects, data scientists, and business analysts to understand data requirements and translate them into technical specifications.
- Participate actively in an agile development environment, including stand-ups, sprint planning, and retrospectives.
- Communicate effectively with users, stakeholders, and team members to gather requirements, provide updates, and resolve issues.
- Troubleshoot and resolve data-related issues and performance bottlenecks in a timely manner.
- Strong proficiency in Python and/or Apache Spark for data processing and ETL development.
- Strong SQL knowledge, with proven experience in writing complex queries, stored procedures, and optimizing database performance.
Apply by
Details last changed . About these details
Report this job
