Microsoft Fabric & Airflow Engineer
- Brampton, ON, Canada
Contract
Applications open in a new tab.
Job details
- Job type
- Contract
- Apply by
- Posting language
- English
- Posted
Original job description
Position Name – Microsoft Fabric & Airflow Engineer
Type of hiring – Subcon
Location – Brampton, ON (Onsite)
Job Description:
Responsibilities:
- Lead the offshore engineering pod, providing mentorship and guidance to team members.
- Design, build, and optimize data pipelines in Fabric Data Factory.
- Orchestrate workloads using Fabric Airflow to ensure efficient data processing.
- Manage Lakehouse and Warehouse items in OneLake, including configuration of shortcuts to various cloud storage solutions.
- Implement and maintain Direct Lake semantic model configurations and Oracle mirroring setups.
- Utilize Spark, PySpark, SQL, T SQL, and Delta Lake for data processing and transformation.
- Conduct interoperability testing with Databricks and Snowflake to ensure seamless integration.
- Utilize Azure DevOps and Git for version control and CI/CD of Fabric artifacts.
- Monitor consumption using the Fabric Capacity Metrics app and perform disciplined benchmarking.
- Document technical processes and maintain clean technical documentation.
- Coordinate across time zones with an onsite architect to ensure project alignment.
Mandatory Skills:
- Strong experience with Data Fabric technologies.
- Proficient in building and tuning pipelines in Fabric Data Factory.
- Hands on experience with Fabric Airflow for workload orchestration.
- Expertise in managing Lakehouse and Warehouse items in OneLake.
- Solid understanding of cloud storage solutions: ADLS Gen2, AWS S3, and GCP Cloud Storage.
- Proficient in Spark, PySpark, SQL, T SQL, and Delta Lake.
- Experience with Direct Lake semantic model configuration and Oracle mirroring setup.
- Familiarity with Databricks and Snowflake for interoperability testing.
- Fluency in Azure DevOps and Git for version control and CI/CD.
- Experience using the Fabric Capacity Metrics app for monitoring.
Preferred Skills:
- Experience in mentoring and leading engineering teams.
- Strong analytical and problem-solving skills.
- Excellent communication skills, both verbal and written.
More about this job
Responsibilities
Lead and mentor an offshore engineering pod while designing, building, and optimizing Fabric Data Factory pipelines and orchestrating workloads with Fabric Airflow. Manage OneLake Lakehouse and Warehouse items, implement semantic models and Oracle mirroring, test interoperability, maintain CI/CD and monitoring, document processes, and coordinate with an onsite architect.
Requirements
Requires strong Microsoft Fabric experience, including Fabric Data Factory, Fabric Airflow, OneLake, and cloud storage solutions, along with proficiency in Spark, PySpark, SQL, T-SQL, and Delta Lake. Candidates should also have experience with Direct Lake semantic models, Oracle mirroring, Databricks and Snowflake interoperability, Azure DevOps, Git, and Fabric capacity monitoring.
Skills
- Microsoft Fabric
- Fabric Data Factory
- Fabric Airflow
- OneLake
- Lakehouse
- Warehouse
- ADLS Gen2
- AWS S3
- Google Cloud Storage
- Spark
- PySpark
- SQL
- T-SQL
- Delta Lake
- Direct Lake Semantic Models
- Azure DevOps
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
- Management & Leadership
Keywords
- Microsoft Fabric
- Fabric Data Factory
- Fabric Airflow
- OneLake
- Lakehouse
- Warehouse
- ADLS Gen2
- AWS S3
- Google Cloud Storage
- Spark
- PySpark
- SQL
- T-SQL
- Delta Lake
- Direct Lake
- Semantic Models
- Oracle Mirroring
- Databricks
- Snowflake
- Azure DevOps
- Git
- CI/CD
- Fabric Capacity Metrics App
- Data Pipelines
- Workload Orchestration
- Interoperability Testing
- Benchmarking
- Technical Documentation
- Engineering Leadership
- Telecommunications