Databricks Architect
- Toronto, ON, Canada
Full-time
Applications open in a new tab.
Job details
- Job type
- Full-time
- Apply by
- Posting language
- English
- Posted
Original job description
- Experience Required: 10 & Above
- Design and implement scalable data platforms using Databricks Lakehouse| Delta Lake| Spark| and PySpark.
- Architect batch and real-time data pipelines on Azure/AWS/GCP.
- Lead ETL modernization and migration from Informatica PowerCenter/IICS/IDMC to Databricks.
- Define data governance| security| performance optimization| and best practices.
- Collaborate with business and technical teams to deliver enterprise data solutions.
- Strong expertise in Databricks| Spark| PySpark| SQL| Delta Lake| Unity Catalog.
- Hands-on experience with Informatica PowerCenter and/or IICS/IDMC.
- Experience in data warehousing| data modeling| and cloud platforms (Azure preferred).
- Knowledge of CI/CD| Git| and Agile methodologies.Preferred Skills:-Databricks and Cloud certifications.Experience with Snowflake| Azure Synapse| Kafka| or Data Governance tools.
More about this job
Responsibilities
Design and implement scalable data platforms using Databricks Lakehouse and architect batch and real-time data pipelines. Lead the modernization and migration of ETL processes from Informatica to Databricks while defining data governance and security best practices.
Requirements
Requires over 10 years of experience with strong expertise in Databricks, Spark, and cloud platforms, preferably Azure. Candidates must have hands-on experience with Informatica PowerCenter or IICS/IDMC and knowledge of CI/CD and Agile methodologies.
Skills
- Databricks
- Spark
- PySpark
- SQL
- Delta Lake
- Unity Catalog
- Informatica PowerCenter
- IICS
- IDMC
- Data Warehousing
- Data Modeling
- Azure
- CI/CD
- Git
- Agile
Visa sponsorship
Not detected in the job text
Categories
- Data & Analytics
- Technology
- Software
- Consulting
- Engineering
Keywords
- Databricks
- Lakehouse
- Delta Lake
- Spark
- PySpark
- Azure
- AWS
- GCP
- ETL
- Informatica PowerCenter
- IICS
- IDMC
- Data Governance
- Performance Optimization
- SQL
- Unity Catalog
- Data Warehousing
- Data Modeling
- CI/CD
- Git
- Agile
- Snowflake
- Azure Synapse
- Kafka