Lead Software Engineer
- Halifax, NS, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Experience
- At least 8 years of IT experience with minimum 5 years of experience in below skills:
- Posting language
- English
- Posted
Original job description
Position Summary
The SDET (Software Development Engineer in Test) is responsible for ensuring the quality
and reliability of ETL processes and data integration workflows. This role involves designing
and implementing automated tests to validate data transformation and migration, as well as
collaborating with data engineers and developers to identify and resolve data quality issues.
The ideal candidate will leverage their expertise in ETL testing and data validation to
enhance the overall data pipeline performance and integrity.
Minimum Qualification
At least 8 years of IT experience with minimum 5 years of experience in below skills:
Tech Skills
Proficiency in ETL Testing and developing automation framework &
testcases in atleast one scripting language.
Strong SQL skills, with experience in handling big data using ETL tools (e.g.,
Informatica).
Mandatory work experience with PySpark
Testing experience in cloud environment (AWS)
S3, SSS, Glue, EMR, Data Catalog, Crawler, SQS, ETL Jobs, Event Bridge.
Athena, Redshift Spectrum Databases.
AWS Commands for running the jobs, file process and movements.
Experience with Test Automation and DevOps practices in ETL and Cloud Systems.
Experience with Test Automation and DevOps practices in ETL and Cloud
Work experience in a Big Data ecosystem processing data in data
lakehouse.
Familiarity with DBT (Data Build Tool) or similar data transformation tools is a
plus.
Experience providing testing oversight in Agile team.
Should have excellent business communication to interact with Business teams and
cross-functional teams.
Responsibilities
Review requirements, specifications, and technical design documents to understand
and feedback.
Experience in developing and implementing test strategies for complex initiatives.
Estimate, prioritize, plan and coordinate testing activities.
Create automation test framework.
Design, develop and execute automation scripts.
Identify, record, document thoroughly and track bugs.
Strong problem-solving skills, troubleshooting, and root cause analysis skills.
Perform thorough regression testing when bugs are resolved.
Develop and apply testing processes for new and existing features to meet client
needs.
More about this job
Responsibilities
Develop and execute automated tests and testing strategies to validate ETL processes, data transformations, migrations, and cloud-based data workflows. Coordinate testing activities, track and troubleshoot defects, perform regression testing, and collaborate with business, data engineering, and development teams to improve data quality and reliability.
Requirements
Requires at least 8 years of IT experience, including at least 5 years in relevant skills, with strong ETL testing, automation, SQL, PySpark, AWS, and big data experience. Candidates should also have experience with cloud data services, DevOps and Agile practices, and strong communication and problem-solving skills; familiarity with DBT is a plus.
Skills
- ETL Testing
- Test Automation
- SQL
- PySpark
- AWS
- Informatica
- Big Data
- Data Validation
- Data Integration
- Data Lakehouse
- DevOps
- Agile
- DBT
- Regression Testing
- Root Cause Analysis
Visa sponsorship
Not detected in the job text
Categories
- Software
- Technology
- Data & Analytics
Keywords
- ETL Testing
- Test Automation
- Automation Framework
- SQL
- PySpark
- AWS
- Informatica
- S3
- AWS Glue
- Amazon EMR
- AWS Data Catalog
- AWS Glue Crawler
- Amazon SQS
- Amazon EventBridge
- Amazon Athena
- Amazon Redshift Spectrum
- AWS Commands
- Big Data
- Data Lakehouse
- Data Integration
- Data Transformation
- Data Migration
- Data Validation
- Data Quality
- DevOps
- Agile
- DBT
- Regression Testing
- Root Cause Analysis
- Test Strategy