Senior Databricks Platform Architect / Lead
- Ottawa, ON, Canada
$150,000 to $250,000 CAD / year · Contract
Applications open in a new tab.
Job details
- Pay
- $150,000 to $250,000 CAD / year
- Job type
- Contract
- Apply by
- Experience
- A minimum of 10 years of professional experience working in the IT industry or performing IT-related roles
- Posting language
- English
- Posted
Original job description
Please note that these opportunities are currently prospective and are not associated with a guaranteed placement or awarded contract. Candidates who join our network will be considered for relevant opportunities as they become available.
49 Solutions is seeking a Senior Databricks Platform Architect / Lead to provide technical leadership for the design and implementation of secure, enterprise-scale Databricks environments supporting Government of Canada data, analytics and AI initiatives.
Responsibilities
- Lead the architecture and implementation of enterprise Databricks platforms within Microsoft Azure
- Define target-state Lakehouse, workspace and Unity Catalog architectures
- Design secure foundations supporting data engineering, Databricks SQL, BI, analytics, AI and machine learning workloads
- Establish architectural patterns for Delta Lake, Lakeflow, Databricks SQL and Unity Catalog
- Design integration with cloud, enterprise and on-premises data sources
- Work with Azure cloud and network specialists on VNet injection, Private Link, routing and secure connectivity
- Define identity and access patterns using Microsoft Entra ID, SCIM and role-based access controls
- Guide platform security approval, governance, classification, lineage, auditing and cost controls
- Provide technical direction to data engineering, DevOps, security, networking and governance resources
- Lead discovery sessions, architecture workshops and design reviews with federal stakeholders
- Develop reusable architectures, implementation standards, runbooks and knowledge-transfer materials
Mandatory Qualifications
- A minimum of 10 years of professional experience working in the IT industry or performing IT-related roles
- A valid and active Government of Canada Reliability, Secret or Top Secret security clearance
- Significant experience designing enterprise data, cloud or analytics platforms
- Demonstrated hands-on Databricks and Microsoft Azure experience
- Experience with Databricks workspaces, Delta Lake, Unity Catalog and enterprise ETL/ELT architecture
- Strong knowledge of cloud networking, identity, security and data governance
- Experience leading complex technical delivery and preparing architecture documentation
- Strong client-facing consulting and communication skills
- Databricks Lakehouse
- Unity Catalog
- Delta Lake
- Lakeflow
- Databricks SQL
- Apache Spark / PySpark
- Python and SQL
- Microsoft Azure
- Microsoft Entra ID
- Cloud networking and security
- Enterprise data architecture
- Data governance
- CI/CD and DevOps concepts
- Government of Canada Databricks implementations
- Protected B, ITSG-33, PBMM, SA&A or ATO
- Microsoft Purview
- Terraform and Databricks Asset Bundles
- Migration from SAS, Synapse, DataStage, DB2, Oracle, Teradata or Netezza
- Databricks AI/BI, Genie, MLflow or Mosaic AI
- Bilingualism in English and French
Target Certification
- Databricks Certified Data Engineer Professional or another advanced Databricks certification directly relevant to solution architecture
Additional Asset Certifications
- Microsoft Certified: Azure Solutions Architect Expert
- Databricks Solutions Architect training or accreditation
- Databricks Platform Administrator accreditation
- Databricks Data & AI Governance with Unity Catalog training
- Databricks EDW/ETL migration training
- TOGAF or equivalent
Candidates must hold a valid and active Government of Canada Reliability, Secret or Top Secret security clearance.
Location
Ottawa, Ontario, Canada. Engagements may be remote, hybrid, or performed at a Government of Canada location. Candidates located in Toronto or Montreal are also encouraged to apply, provided they are willing and able to travel to Ottawa for onsite meetings approximately once per month, as required.
Work arrangements will depend on the client, project and security requirements.
Use of Artificial Intelligence and Applicant Privacy
Use of Artificial Intelligence (AI tools): 49 Solutions may use approved, secure, and private enterprise AI tools, including Microsoft 365 Copilot, to support recruitment and proposal activities. Information processed using our approved enterprise AI tools is not used to train public AI models or underlying foundation models. Personal contact information (address, email address and phone numbers) will be removed before uploading into the enterprise AI environment. Personal Information shall not be entered into public or consumer AI systems, and our use of enterprise AI is subject to applicable privacy requirements, including PIPEDA, as well as organizational security, retention, and deletion policies.
By submitting an application or CV to 49 Solutions, you consent to the collection and use of your information for staffing, recruitment, proposal, contracting and placement purposes. AI use for these purposes is optional. If you prefer that your CV not be processed using AI, please let us know. You can opt out of AI use and still apply for opportunities, remain in our candidate database, work with 49 Solutions, and be considered for current and future opportunities.
Consent may be withdrawn at any time, subject to applicable legal, contractual, operational, and record-keeping requirements. For more information, our full Privacy Policy is available upon request.
More about this job
Responsibilities
Lead the architecture and implementation of secure, enterprise-scale Databricks platforms on Azure, including Lakehouse, workspace, Unity Catalog, networking, identity, security, governance, and data integration designs. Provide technical direction to delivery teams and federal stakeholders, and produce reusable architecture standards, implementation guidance, runbooks, and knowledge-transfer materials.
Requirements
Applicants must have at least 10 years of IT experience, an active Government of Canada Reliability, Secret, or Top Secret clearance, and substantial hands-on experience designing enterprise data or cloud platforms with Databricks and Azure. They must also demonstrate expertise in Databricks architecture, cloud networking, identity, security, governance, technical leadership, architecture documentation, and client-facing communication.
Skills
- Databricks
- Microsoft Azure
- Lakehouse Architecture
- Unity Catalog
- Delta Lake
- Lakeflow
- Databricks SQL
- Apache Spark
- PySpark
- Python
- SQL
- Cloud Networking
- Cloud Security
- Data Governance
- Enterprise Data Architecture
- CI/CD
Visa sponsorship
Not detected in the job text
Categories
- Technology
- Data & Analytics
- Software
- Management & Leadership
- Government & Public Sector
Keywords
- Databricks
- Microsoft Azure
- Lakehouse
- Unity Catalog
- Delta Lake
- Lakeflow
- Databricks SQL
- Apache Spark
- PySpark
- Python
- SQL
- Microsoft Entra ID
- SCIM
- Role-Based Access Control
- VNet Injection
- Private Link
- Cloud Networking
- Data Governance
- Enterprise Data Architecture
- ETL/ELT
- CI/CD
- DevOps
- Government of Canada Security Clearance
- Protected B
- ITSG-33
- PBMM
- Security Assessment and Authorization
- Microsoft Purview
- Terraform
- Databricks Asset Bundles