GCP Senior Data Engineer
- Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Posting language
- English
- Posted
Original job description
Job Description:
We are looking for an experienced GCP Data Engineering Lead to lead a team of 10 engineers and design, develop, and optimize scalable data engineering solutions on Google Cloud Platform (GCP).
Key Responsibilities:
- Lead and mentor onshore and offshore data engineering teams.
- Design and optimize ETL/ELT pipelines using Dataflow, Apache Beam, Dataproc, Spark, and Pub/Sub.
- Manage BigQuery, Cloud Storage, Data Fusion, and Cloud Composer.
- Establish coding standards, code reviews, CI/CD, and monitoring practices.
- Optimize data pipeline performance, query efficiency, and cloud costs.
- Implement data quality, governance, metadata, and lineage solutions.
- Collaborate with data architects, analysts, and data scientists to support analytics and AI initiatives.
Required Skills:
- Strong experience with GCP data engineering services, including BigQuery, Dataflow, Dataproc, Pub/Sub, and Composer.
- Proficiency in SQL, Python, and Java/Scala.
- Experience with Apache Spark, Apache Beam, distributed systems, and real-time data processing.
- Hands-on experience with Terraform, Git, CI/CD, and infrastructure as code.
- Experience leading onshore and offshore data engineering teams, preferably managing 10 engineers.
- Valid/relevant GCP certification, such as Professional Data Engineer.
- Exposure to Vertex AI or multi-cloud environments.
More about this job
Responsibilities
Lead and mentor a team of onshore and offshore data engineers while designing, developing, and optimizing scalable GCP data solutions and ETL/ELT pipelines. Establish engineering standards and practices, improve pipeline performance and cloud costs, implement data quality and governance capabilities, and support analytics and AI initiatives.
Requirements
Requires strong experience with GCP data engineering services and proficiency in SQL, Python, and Java or Scala, along with expertise in distributed systems, Spark, Beam, and real-time processing. Candidates should have hands-on Terraform, Git, CI/CD, and infrastructure-as-code experience, leadership experience managing engineering teams, and a relevant GCP certification; exposure to Vertex AI or multi-cloud environments is also noted.
Skills
- Google Cloud Platform
- BigQuery
- Dataflow
- Dataproc
- Pub/Sub
- Cloud Composer
- Cloud Storage
- Data Fusion
- SQL
- Python
- Java
- Scala
- Apache Spark
- Apache Beam
- Terraform
- CI/CD
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Management & Leadership
- Finance & Accounting
Keywords
- Google Cloud Platform
- GCP
- Data Engineering
- BigQuery
- Dataflow
- Dataproc
- Pub/Sub
- Cloud Composer
- Cloud Storage
- Data Fusion
- ETL/ELT
- Apache Beam
- Apache Spark
- Distributed Systems
- Real-Time Data Processing
- SQL
- Python
- Java
- Scala
- Terraform
- Git
- CI/CD
- Infrastructure as Code
- Data Quality
- Data Governance
- Metadata
- Data Lineage
- Vertex AI
- Banking