We are seeking an ML Dev Engineer with expertise in Databricks Unity Catalog and Feature Store to build and maintain scalable data and machine learning foundations. The role focuses on data organization, access management, feature engineering, and supporting production ML workflows.
Key Responsibilities
- Set up and manage Databricks Unity Catalog for data organization and secure access control.
- Design and operationalize Feature Stores to support production ML models.
- Build and maintain efficient data/ETL pipelines to process and serve ML features.
- Integrate data solutions across Databricks, Azure Cosmos DB, ACR, and other Azure services.
- Monitor, troubleshoot, and optimize pipeline and Feature Store performance.
- Collaborate with data engineering and ML teams to support production workflows.
Required Skills & Experience
- Strong hands-on experience with Databricks Unity Catalog, including data asset management and access control.
- Experience with Databricks Feature Store or similar solutions.
- Strong knowledge of scalable ETL pipelines in Databricks.
- Proficiency in Python, Spark, and Java.
- Familiarity with Azure Cosmos DB and Azure Container Registry (ACR).
- Understanding of ML workflows and Feature Store architecture.
- Experience with monitoring tools such as Splunk or Datadog.
- Familiarity with AKS for container deployment and management.
- Strong problem-solving and collaboration skills.
