Job Description :
- Audit the current Databricks workspace state, including environments, clusters, Unity Catalog setup, access controls, and cost configuration.
- Implement and harden Unity Catalog, including namespace structure, access controls, cluster policies, and environment separation.
- Define workspace standards, naming conventions, access request workflows, and user/team onboarding processes.
- Establish data quality frameworks, including SLAs, certification criteria, and data ownership models.
- Build automation templates for routine workspace operations such as cluster lifecycle management, cost alerting, and utilization monitoring.
- Stand up observability dashboards covering platform health, cost, job success rates, and DQ SLA KPIs, and monitor core data asset lineage across workspaces.
- Create runbooks for recurring operational tasks that can be executed by a mid-level/junior engineer without ongoing support from the senior resource.
- Conduct knowledge-transfer sessions with the platform engineering team.
- Provide final documentation covering architecture decisions, operational standards, open items, and recommendations for the permanent hire profile.
Key Responsabilities.
- Evaluate and optimize Databricks environments, including workspaces, clusters, catalogs, permissions, and security configurations.
- Define and implement governance standards to ensure secure, scalable, and compliant platform operations.
- Lead Unity Catalog adoption, designing data structures, access models, and environment segregation strategies.
- Establish RBAC policies, workspace standards, naming conventions, and user access processes.
- Drive data governance initiatives, including data quality frameworks, ownership models, SLAs, lineage, and stewardship practices.
- Develop automation solutions and Infrastructure as Code (IaC) frameworks to streamline platform administration and reduce manual effort.
- Implement cost optimization, monitoring, and operational excellence practices across Databricks environments.
- Design observability and performance dashboards to track platform health, utilization, job success rates, and data quality KPIs.
- Create operational documentation, runbooks, and support procedures to improve platform reliability and team readiness.
- Partner with security, compliance, infrastructure, and data engineering teams to align platform capabilities with business needs.
- Provide training, mentoring, and knowledge transfer sessions to empower platform administrators and engineering teams.
- Recommend platform improvements and long-term strategies to enhance governance, scalability, and organizational maturity.
Must have skills: Azure Data Factory, Databricks, Unity Catalog
Good to have skills: Data platform implementation, Google Workspace Administration