GCP Senior Data Engineer
- Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Posting language
- English
- Posted
Original job description
GCP Expertise: Hands-on experience with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer (Airflow), Cloud Storage, and Terraform.
Key Responsibilities
- Design, build, and optimize batch and real-time data pipelines.
- Develop ETL/ELT solutions using Dataflow, Spark, Pub/Sub, and BigQuery.
- Architect scalable and cost-efficient GCP data platforms.
- Lead and mentor a team of 10 data engineers.
- Establish coding, CI/CD, monitoring, data quality, and governance standards.
- Optimize performance, reliability, and GCP costs.
- Collaborate with architects, data scientists, and analysts on analytics/ML solutions.
More about this job
Responsibilities
Design, build, and optimize batch and real-time data pipelines and scalable, cost-efficient GCP data platforms using tools such as Dataflow, Spark, Pub/Sub, and BigQuery. Lead and mentor a team of 10 data engineers, establish engineering and governance standards, optimize platform performance and costs, and collaborate on analytics and machine-learning solutions.
Requirements
Requires strong proficiency in SQL, Python, Java or Scala, Spark, and Apache Beam, along with hands-on experience across GCP data services and Terraform. The role calls for senior-level data engineering capability; a GCP Professional Data Engineer certification, Vertex AI, dbt, real-time processing, and multi-cloud experience are nice to have.
Skills
- SQL
- Python
- Java
- Scala
- Spark
- Apache Beam
- BigQuery
- Dataflow
- Dataproc
- Pub/Sub
- Composer (Airflow)
- Cloud Storage
- Terraform
- ETL/ELT
- CI/CD
- Data Governance
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Management & Leadership
Keywords
- SQL
- Python
- Java
- Scala
- Spark
- Apache Beam
- Google Cloud Platform (GCP)
- BigQuery
- Dataflow
- Dataproc
- Pub/Sub
- Cloud Composer
- Airflow
- Cloud Storage
- Terraform
- Batch Data Pipelines
- Real-Time Data Processing
- ETL/ELT
- Data Platform Architecture
- CI/CD
- Monitoring
- Data Quality
- Data Governance
- Cost Optimization
- Vertex AI
- dbt
- Multi-Cloud
- GCP Professional Data Engineer
- Analytics
- Machine Learning