About the Role
We are looking for a hands-on Data Engineering Lead to run a team of 10 engineers, both onshore and offshore, building cloud data platforms on Google Cloud. This is a leadership role, not a senior individual contributor position. You will set technical direction, review code, and make sure the team delivers reliable, cost-efficient data pipelines.
What You'll Do
- Build pipelines: Design and improve ETL/ELT pipelines using Dataflow (Apache Beam), Dataproc (Spark), Pub/Sub and Cloud Functions
- Own the data platform: Architect and manage BigQuery, Cloud Storage and Data Fusion for ingesting, transforming and storing data
- Lead the team: Guide a team of 10 engineers, review code, and set standards for data engineering, CI/CD and monitoring (Cloud Monitoring, Logging)
- Tune performance and cost: Improve pipeline speed and query efficiency, and control GCP spend through practices like slot reservations and partitioning
- Partner across teams: Work with data architects, data scientists and analysts to deliver self-service data platforms and support advanced analytics and ML use cases
- Protect data quality: Put data validation, lineage and metadata management in place using tools such as Data Catalog and dbt
Required Skills
- Leadership: Proven experience leading data engineering teams across onshore and offshore locations. Candidates without team leadership experience will not be considered.
- Core technical skills: SQL, Python, Java or Scala, and distributed systems such as Spark and Beam
- GCP expertise: Hands-on work with BigQuery, Dataflow, Dataproc, Pub/Sub, Composer (Airflow) and Terraform
- DevOps: Familiarity with CI/CD, Git and infrastructure as code
Nice to Have
- GCP certification, such as Professional Data Engineer
- Experience with real-time processing, Vertex AI or multi-cloud environments
