Data Bricks Admin/Azure Data Bricks Platform Engineer

Data Bricks Admin/Azure Data Bricks Platform Engineer

Job Description:

  • We are looking for an Data Bricks Administrator with 4+ years of experience to join our team for one of UKs most well-known business partner.
  • You will use various methods to transform raw data into useful data systems.
  • Overall, you’ll strive for efficiency by aligning data systems with business goals.
  • The Azure Databricks Platform Engineer will be responsible for providing technical expertise in managing Azure Databricks platform and deploying applications on it.
  • The candidate must be self-motivated and apply knowledge of Azure to drive solutions, support the development team, and create documentation to support and describe technical solutions.
  • An ideal candidate will have deep technical Azure platform expertise and demonstrate high standards of accountability, customer focus, and quality.

Responsibilities:

  • Implement, setup, configure, monitor, and maintain various Microsoft Azure Databricks solutions including: o Databricks access control o Databricks Cost Management o Databricks Performance Management. o Azure Application Gateways o Databricks Backup (IaC based) o Azure virtual network so Databricks Storage (DBFS)o Azure Storage (ADLS) o Databricks SQL o Disaster Recovery Deploy and maintain PaaS & SaaS Databricks platform for Machine Learning (ML) development, QA, staging, and production environments hosted on Azure cloud infrastructure.
  • Understanding of a multitude of Azure resources and services such as containers, virtual machines, storage, firewalls, network security groups, web application firewalls, vnets, log analytics, sentinel, azure monitor, alerting, metrics, Data Factory, Databricks, Azure AD, Data Catalog, Web Apps, Functions, Application Service Environments, Key Vaults, Azure Backups. Manage Databricks environments in line with security guidelines.
  • Develop Infrastructure as Code(IaC) to automate databricks environment deployments.
  • Aid developers teams with implementing automation scripts that follow the company standards.
  • Evaluate and automate the scaling and capacity requirements within Azure environments. Create custom scripts to automate support processes where applicable.
  • Manage Databricks infrastructure and collaborate with various teams to ensure it integrates seamlessly with all components including network, security, messaging, and remote access.
  • Troubleshoot and resolve Databricks environment performance issues, connectivity issues, security issues.
  • Solid experience in the administration and performance tuning of Databricks is required for success at this position.
  • Working knowledge of Azure Active Directory, Azure Cloud components is a PLUS.
  • Solid scripting skills is required for success at this position.
  • Experience in working with formal Change Control Systems. Mentor, coach, and share knowledge with other team members throughout the organization.

Additional Must Have Skills:

  • Good conversion of high-level business & technical requirements into technical specs.
  • Feeling comfortable in using Azure cloud technologies.
  •  Customer-centric, passionate about delivering great digital products and services.
  • Demonstrating true engineering craftsmanship mindset.
  • Passionate about continuous improvement, collaboration, and great teams.
  •  Strong problem-solving skills coupled with good communication skills.
  • Open minded, inquisitive, life-long learner.