GCP Senior Data Engineer
- Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Posting language
- English
- Posted
Original job description
- Leadership: Experience leading data engineering teams onshore and offshore. This is a lead position, not a senior data engineer position. Candidates without leadership experience will not be considered for this role.
- Technical: Proficiency in SQL, Python, Java/Scala, and distributed systems (Spark, Beam).
- GCP Expertise: Hands-on with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer (Airflow), and Terraform.
- DevOps: Familiarity with CI/CD, Git, and infrastructure-as-code.
- GCP Certifications (e.g., Professional Data Engineer).
- Experience with real-time processing, Vertex AI, or multi-cloud environments.
- Pipeline Development: Build and optimize ETL/ELT pipelines using Dataflow (Apache Beam), Dataproc (Spark), Pub/Sub, and Cloud Functions.
- Data Infrastructure: Architect and manage BigQuery, Cloud Storage, and Data Fusion for efficient data ingestion, transformation, and storage.
- Team Leadership: Lead a team of 10 engineers, review code, establish data engineering standards, CI/CD, and monitoring (e.g., Cloud Monitoring, Logging).
- Performance & Cost: Optimize pipeline performance, query efficiency, and GCP cost management (e.g., slot reservations, partitioning).
- Collaboration: Work with data architects, scientists, and analysts to deliver self-service data platforms and support advanced analytics/ML use cases.
- Data Quality & Governance: Implement data validation, lineage, and metadata management (e.g., Data Catalog, dbt).
More about this job
Responsibilities
Lead a team of 10 engineers while building and optimizing data pipelines and managing GCP data infrastructure, including BigQuery, Cloud Storage, and Data Fusion. Establish engineering standards and monitoring, optimize performance and costs, and collaborate on self-service data platforms, analytics, machine learning, data quality, and governance.
Requirements
Candidates must have experience leading onshore and offshore data engineering teams; this is a lead role, and leadership experience is mandatory. Required technical expertise includes SQL, Python, Java or Scala, distributed systems, GCP data services, Terraform, and CI/CD; GCP certification and experience with real-time processing, Vertex AI, or multi-cloud environments are preferred.
Skills
- Team Leadership
- SQL
- Python
- Java
- Scala
- Distributed Systems
- Apache Spark
- Apache Beam
- BigQuery
- Dataflow
- Dataproc
- Pub/Sub
- Cloud Composer
- Terraform
- CI/CD
- ETL/ELT
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Management & Leadership
- Engineering
Keywords
- Data Engineering
- Team Leadership
- Onshore And Offshore Teams
- SQL
- Python
- Java
- Scala
- Distributed Systems
- Apache Spark
- Apache Beam
- Google Cloud Platform
- BigQuery
- Dataflow
- Dataproc
- Pub/Sub
- Cloud Composer
- Airflow
- Terraform
- CI/CD
- Git
- Infrastructure As Code
- ETL/ELT
- Cloud Storage
- Data Fusion
- Cloud Functions
- Cloud Monitoring
- Cloud Logging
- Data Catalog
- dbt
- Vertex AI