Closed listing
Microsoft Fabric & Airflow Engineer
- Brampton, ON, Canada
Contract
This opening has closed.
The original listing is kept below for reference.
Job details
- Job type
- Contract
- Closing date
- Posting language
- English
- Posted
Similar jobs hiring now
Search current openingsOriginal job description
Archived listing. The details below describe a past opening.
Position Name – Microsoft Fabric & Airflow Engineer
Type of hiring – Subcon
Location – Brampton, ON (Onsite)
Job Description:
Responsibilities:
- Lead the offshore engineering pod, providing mentorship and guidance to team members.
- Design, build, and optimize data pipelines in Fabric Data Factory.
- Orchestrate workloads using Fabric Airflow to ensure efficient data processing.
- Manage Lakehouse and Warehouse items in OneLake, including configuration of shortcuts to various cloud storage solutions.
- Implement and maintain Direct Lake semantic model configurations and Oracle mirroring setups.
- Utilize Spark, PySpark, SQL, T SQL, and Delta Lake for data processing and transformation.
- Conduct interoperability testing with Databricks and Snowflake to ensure seamless integration.
- Utilize Azure DevOps and Git for version control and CI/CD of Fabric artifacts.
- Monitor consumption using the Fabric Capacity Metrics app and perform disciplined benchmarking.
- Document technical processes and maintain clean technical documentation.
- Coordinate across time zones with an onsite architect to ensure project alignment.
Mandatory Skills:
- Strong experience with Data Fabric technologies.
- Proficient in building and tuning pipelines in Fabric Data Factory.
- Hands on experience with Fabric Airflow for workload orchestration.
- Expertise in managing Lakehouse and Warehouse items in OneLake.
- Solid understanding of cloud storage solutions: ADLS Gen2, AWS S3, and GCP Cloud Storage.
- Proficient in Spark, PySpark, SQL, T SQL, and Delta Lake.
- Experience with Direct Lake semantic model configuration and Oracle mirroring setup.
- Familiarity with Databricks and Snowflake for interoperability testing.
- Fluency in Azure DevOps and Git for version control and CI/CD.
- Experience using the Fabric Capacity Metrics app for monitoring.
Preferred Skills:
- Experience in mentoring and leading engineering teams.
- Strong analytical and problem-solving skills.
- Excellent communication skills, both verbal and written.
More about this job
Responsibilities
Lead and mentor an offshore engineering pod while designing, building, and optimizing Fabric Data Factory pipelines and orchestrating workloads with Fabric Airflow. Manage OneLake Lakehouse and Warehouse items, configure Direct Lake and Oracle mirroring, test interoperability with Databricks and Snowflake, monitor capacity, maintain documentation, and coordinate with an onsite architect.
Requirements
Candidates need strong Microsoft Fabric experience, including pipeline development and tuning, Fabric Airflow orchestration, and OneLake management. They should be proficient in Spark, PySpark, SQL, T-SQL, and Delta Lake, with experience in cloud storage, Direct Lake, Oracle mirroring, Databricks and Snowflake interoperability, Azure DevOps, Git, and Fabric capacity monitoring.
Skills
- Microsoft Fabric
- Fabric Data Factory
- Fabric Airflow
- OneLake
- Lakehouse
- Data Warehouse
- ADLS Gen2
- AWS S3
- Google Cloud Storage
- Spark
- PySpark
- SQL
- T-SQL
- Delta Lake
- Azure DevOps
- Git
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
- Management & Leadership
Keywords
- Microsoft Fabric
- Fabric Data Factory
- Fabric Airflow
- OneLake
- Lakehouse
- Warehouse
- ADLS Gen2
- AWS S3
- Google Cloud Storage
- Spark
- PySpark
- SQL
- T-SQL
- Delta Lake
- Direct Lake
- Semantic Models
- Oracle Mirroring
- Databricks
- Snowflake
- Azure DevOps
- Git
- CI/CD
- Fabric Capacity Metrics
- Data Pipelines
- Workload Orchestration
- Interoperability Testing
- Benchmarking
- Technical Documentation
- Engineering Leadership