Microsoft Fabric & Airflow Resource
- Brampton, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Experience
- Need Tech Lead or Architect with minimum 10 Years of overall experience.
- Posting language
- English
- Posted
Original job description
Responsibilities
- Lead the offshore engineering pod, providing mentorship and guidance to team members.
- Design, build, and optimize data pipelines in Fabric Data Factory.
- Orchestrate workloads using Fabric Airflow to ensure efficient data processing.
- Manage Lakehouse and Warehouse items in OneLake, including configuration of shortcuts to various cloud storage solutions.
- Implement and maintain Direct Lake semantic model configurations and Oracle mirroring setups.
- Utilize Spark, PySpark, SQL, T SQL, and Delta Lake for data processing and transformation.
- Conduct interoperability testing with Databricks and Snowflake to ensure seamless integration.
- Utilize Azure DevOps and Git for version control and CI/CD of Fabric artifacts.
- Monitor consumption using the Fabric Capacity Metrics app and perform disciplined benchmarking.
- Document technical processes and maintain clean technical documentation.
- Coordinate across time zones with an onsite architect to ensure project alignment.
- Strong experience with Data Fabric technologies.
- Proficient in building and tuning pipelines in Fabric Data Factory.
- Hands on experience with Fabric Airflow for workload orchestration.
- Expertise in managing Lakehouse and Warehouse items in OneLake.
- Solid understanding of cloud storage solutions: ADLS Gen2, AWS S3, and GCP Cloud Storage.
- Proficient in Spark, PySpark, SQL, T SQL, and Delta Lake.
- Experience with Direct Lake semantic model configuration and Oracle mirroring setup.
- Familiarity with Databricks and Snowflake for interoperability testing.
- Fluency in Azure DevOps and Git for version control and CI/CD.
- Experience using the Fabric Capacity Metrics app for monitoring.
- Experience in mentoring and leading engineering teams.
- Strong analytical and problem-solving skills.
- Excellent communication skills, both verbal and written.
More about this job
Responsibilities
Lead and mentor an offshore engineering pod while designing, building, and optimizing Fabric data pipelines and orchestrated workloads. Manage OneLake, semantic models, and Oracle mirroring; test interoperability with Databricks and Snowflake; maintain CI/CD, monitor capacity, document processes, and coordinate with an onsite architect.
Requirements
Requires at least 10 years of overall experience and strong expertise in Microsoft Fabric, including Fabric Data Factory, Fabric Airflow, OneLake, and cloud storage solutions. Candidates should be proficient in Spark, PySpark, SQL, T-SQL, and Delta Lake, with experience in Direct Lake, Oracle mirroring, Databricks, Snowflake, Azure DevOps, Git, and Fabric capacity monitoring.
Skills
- Microsoft Fabric
- Fabric Data Factory
- Fabric Airflow
- OneLake
- Lakehouse
- Data Warehouse
- ADLS Gen2
- AWS S3
- Google Cloud Storage
- Spark
- PySpark
- SQL
- T-SQL
- Delta Lake
- Direct Lake Semantic Models
- Azure DevOps
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
- Management & Leadership
Keywords
- Microsoft Fabric
- Fabric Data Factory
- Fabric Airflow
- OneLake
- Lakehouse
- Warehouse
- ADLS Gen2
- AWS S3
- Google Cloud Storage
- Spark
- PySpark
- SQL
- T-SQL
- Delta Lake
- Direct Lake
- Oracle Mirroring
- Databricks
- Snowflake
- Azure DevOps
- Git
- CI/CD
- Fabric Capacity Metrics
- Data Pipelines
- Workload Orchestration
- Interoperability Testing
- Benchmarking
- Technical Documentation
- Engineering Leadership
- Mentoring