GCP Data Migration Engineer – PySpark & Salesforce Data Migration
- Brampton, ON, Canada
Applications open in a new tab.
Job details
- Posting language
- English
- Posted
Original job description
GCP Data Migration Engineer – PySpark & Salesforce Data Migration Required Skills & Experience: Must Have (4-5 years minimum): • Strong hands-on experience with GCP Dataproc and Apache Spark (PySpark preferred) • Python scripting for custom data transformations and business logic • Proficiency in BigQuery (SQL, table design, partitioning, optimization) • Experience building ETL pipelines in GCP ecosystem • Solid understanding of Salesforce data model and Salesforce APIs (Bulk API, REST API) • Experience with GCS for file handling and cloud storage patterns Preferred: • Exposure to GitLab CI/CD (understanding pipeline YAML, not building from scratch) • Prior Salesforce data migration projects (especially Health Cloud or Life Sciences Cloud) • Knowledge of DLP implementation and PII handling best practices • Healthcare data compliance knowledge (PIPEDA, HIPAA) Technical Depth: • Ability to debug complex Spark jobs and resolve memory/performance issues • Strong SQL skills for data validation and reconciliation • Experience handling large datasets (300GB+ structured, 3TB+ documents) • Comfortable working with 70+ object schemas and complex relationships |
More about this job
Responsibilities
The GCP Data Migration Engineer at Astra North Infoteck will build and debug data pipelines that move large volumes of data from GCP into Salesforce. The role includes writing Python transformations, validating and reconciling data with SQL, and managing file handling in cloud storage.
Requirements
Candidates need 4 to 5 years of hands-on experience with GCP Dataproc, Apache Spark and PySpark, plus BigQuery, Python and ETL pipelines. Knowledge of the Salesforce data model and its Bulk and REST APIs is also required.
Skills
- GCP Dataproc
- Apache Spark
- PySpark
- Python
- BigQuery
- SQL
- BigQuery partitioning and optimization
- ETL pipeline development
- Salesforce data model
- Salesforce Bulk API
- Salesforce REST API
- Google Cloud Storage
- Spark performance tuning
- Data validation and reconciliation
- GitLab CI/CD pipeline YAML
- Data Loss Prevention (DLP)
- PII handling
- PIPEDA
- HIPAA
- Large dataset handling
Categories
- Software
- Technology
- Data & Analytics
Keywords
- GCP
- Google Cloud Platform
- Dataproc
- Apache Spark
- PySpark
- Python
- BigQuery
- SQL
- ETL
- data pipeline
- Salesforce
- Salesforce Bulk API
- Salesforce REST API
- Google Cloud Storage
- GCS
- GitLab CI/CD
- Health Cloud
- Life Sciences Cloud
- data migration
- data engineer
Seniority
Mid level
Tools
- GCP Dataproc
- Apache Spark
- PySpark
- BigQuery
- Salesforce
- GitLab
Languages
- English