Epicareer Might not Working Properly
Learn More

Principal Databricks DevOps Engineer

Salary undisclosed

Apply on


Original
Simplified
Job Description:
We are looking for a Principal Databricks DevOps Engineer with deep technical expertise in AWS, Databricks, and cloud security. This is a hands-on role that will involve building, configuring, and managing Databricks environments on AWS using CDK and Terraform. The successful candidate will be experienced in both infrastructure automation and cloud security best practices, enabling seamless integration of Databricks environments within enterprise-level AWS accounts.
Key Responsibilities:
  1. Databricks Environment Setup and Configuration:
    • Build, configure, and manage Databricks environments using CDK and Terraform.
    • Establish compute configurations and set up new Databricks environments across multiple AWS accounts.
  2. AWS Integration & Security:
    • Leverage in-depth knowledge of AWS services and security best practices to configure Databricks environments securely and efficiently.
    • Integrate Databricks environments with existing AWS CDK components for seamless deployment and configuration at scale.
  3. Access Management and Security Controls:
    • Set up access for downstream applications, system users, and individual users.
    • Implement user access control through systems like Okta and CyberArk to manage permissions effectively.
  4. Automation and Scripting:
    • Utilize Python for scripting tasks, automating deployment pipelines, and creating efficient workflows for DevOps processes.
  5. Collaboration and Problem Solving:
    • Work closely with cross-functional teams to design and implement scalable solutions.
    • Identify and resolve issues promptly, ensuring high availability and performance of the Databricks platform.
Skills and Qualifications:
  • Extensive hands-on experience with AWS, including deployment, configuration, and security practices.
  • Strong expertise in using CDK and Terraform for infrastructure provisioning and automation.
  • Proven experience in setting up and configuring Databricks environments on AWS.
  • In-depth knowledge of access management systems like Okta and CyberArk for secure user access control.
  • Proficient in Python for scripting and automation.
  • Excellent problem-solving skills and the ability to work in a collaborative team environment.
  • Strong communication skills, with the ability to explain complex technical concepts to cross-functional teams.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job