Epicareer Might not Working Properly
Learn More
D

Sr. Architect Databricks -----------12+ years

Salary undisclosed

Apply on


Original
Simplified

Job Title : Sr. Architect Databricks

Location: Boston, MA Onsite Hybrid

Duration: Long term

Job Type : Contract (We can consider - W2 / C2C )

Responsibilities:

We are seeking a Sr. Architect with 10 to 14 years of experience to join our dynamic team. The ideal candidate will have extensive experience in Databricks Workspace Admin Databricks CLI Databricks Unity Catalog Kafka Azure Monitor Azure Data Lake Store Azure Key Vault Azure DevOps Azure Event Hub Python Databricks SQL Databricks Workflows and PySpark. This is a hybrid role with day shifts and no travel required.

Responsibilities

  • Lead the design and implementation of scalable data architectures using Databricks and Azure technologies.
  • Oversee the administration of Databricks Workspace and ensure optimal performance.
  • Provide expertise in Databricks CLI and Unity Catalog for efficient data management.
  • Implement and manage Kafka for real-time data streaming solutions.
  • Utilize Azure Monitor to track and optimize system performance.
  • Manage data storage solutions using Azure Data Lake Store.
  • Ensure data security and compliance with Azure Key Vault.
  • Coordinate with development teams using Azure DevOps for seamless CI/CD pipelines.
  • Integrate and manage event-driven architectures using Azure Event Hub.
  • Develop and maintain data processing workflows using Python and PySpark.
  • Create and optimize SQL queries within Databricks SQL for data analysis.
  • Automate data workflows and pipelines using Databricks Workflows.
  • Collaborate with cross-functional teams to deliver high-quality data solutions.
  • Provide technical guidance and mentorship to junior team members.
  • Drive continuous improvement initiatives to enhance data architecture and processes.
  • Ensure alignment of data solutions with business objectives and goals.
  • Contribute to the companys mission by delivering impactful data solutions that drive business success.

Qualifications

  • Possess a minimum of 10 years of experience in data architecture and related technologies.
  • Demonstrate expertise in Databricks Workspace Admin Databricks CLI and Databricks Unity Catalog.
  • Have hands-on experience with Kafka for real-time data streaming.
  • Show proficiency in using Azure Monitor Azure Data Lake Store and Azure Key Vault.
  • Exhibit strong skills in Azure DevOps for CI/CD pipeline management.
  • Be experienced in Azure Event Hub for event-driven architectures.
  • Have advanced knowledge of Python and PySpark for data processing.
  • Be skilled in writing and optimizing SQL queries within Databricks SQL.
  • Demonstrate ability to automate workflows using Databricks Workflows.
  • Possess excellent problem-solving and analytical skills.
  • Show strong communication and collaboration abilities.
  • Be committed to continuous learning and staying updated with industry trends.
  • Have a proven track record of delivering successful data solutions.

Certifications Required

  • Databricks Certified Professional Data Engineer Microsoft Certified: Azure Solutions Architect Expert
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job