Closed listing
Big Data Developer
- Toronto, ON, Canada
- Greater Toronto Area, Canada, Canada
Contract
This opening has closed.
The original listing is kept below for reference.
Job details
- Job type
- Contract
- Closing date
- Experience
- 5+ years of experience with Big Data technologies used for ETL
- Posting language
- English
- Posted
Similar jobs hiring now
Search current openingsOriginal job description
Archived listing. The details below describe a past opening.
Position: Big Data Developer
Location: Toronto, ON (5 Days Onsite)
Duration: Contract-Long term
Mandatory Skills: Well versed with Data, ETL Processes, SQL and high-level programming languages such Scala, Java, DevSecOps
We're Hiring: Big Data Developer
We’re looking for a Big Data Developer to join an Agile team and build scalable data pipelines, ETL processes and modern data solutions.
What You'll Do
- Design, develop and maintain scalable data ingestion pipelines
- Build and optimize ETL processes and data pipelines
- Work with Snowflake and integrate it with various data sources and BI platforms
- Develop and optimize complex SQL queries, stored procedures and views
- Work with Big Data technologies including Spark, Hadoop and Hive
- Develop solutions using Scala or Java
- Monitor, troubleshoot and optimize data jobs and platform issues
- Ensure data quality, security, reliability and performance
- Create and maintain documentation for data pipelines, transformations and data models
- Collaborate with developers, data engineers and business stakeholders in an Agile environment
- Contribute to CI/CD, DevSecOps and cloud-based data engineering practices
- Leverage GenAI tools to improve development productivity and code generation
What We're Looking For
Required Skills
- 5+ years of experience with Big Data technologies used for ETL
- Hands-on experience with Snowflake
- Strong SQL and data modeling skills
- Experience with Spark, Hadoop and/or Hive
- Programming experience with Scala or Java
- Experience developing or supporting data pipelines / ETL processes
- Experience with ETL tools such as Informatica, Talend or Apache Airflow
- Experience with at least one cloud platform, preferably AWS
- Understanding of SCM, CI/CD and DevSecOps
- Strong analytical, troubleshooting and communication skills
Nice to Have
- Apache Airflow / workflow orchestration
- Git / GitHub / GitHub Actions
- Automated testing
- Docker / Kubernetes / Containers / OCP4
- Shell scripting
- Infrastructure-as-Code
- Python or other scripting languages
- GenAI / AI-assisted development experience
More about this job
Responsibilities
Design, develop, and maintain scalable data ingestion pipelines and ETL processes using Snowflake and Big Data technologies. Collaborate with stakeholders in an Agile environment to optimize data jobs and ensure data quality and security.
Requirements
Requires over 5 years of experience with Big Data ETL, Snowflake, and strong SQL skills. Proficiency in Scala or Java and experience with cloud platforms like AWS is essential.
Skills
- Big Data
- ETL
- SQL
- Scala
- Java
- DevSecOps
- Snowflake
- Spark
- Hadoop
- Hive
- AWS
- Informatica
- Talend
- Apache Airflow
- Data Modeling
- CI/CD
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Engineering
- Consulting
Keywords
- Big Data
- ETL
- SQL
- Scala
- Java
- DevSecOps
- Snowflake
- Spark
- Hadoop
- Hive
- AWS
- Informatica
- Talend
- Apache Airflow
- Data Modeling
- CI/CD
- Agile
- GenAI
- Git
- GitHub Actions
- Docker
- Kubernetes
- OCP4
- Shell Scripting
- Infrastructure-as-Code
- Python
- Data Pipelines
- Stored Procedures
- Cloud Computing