Databricks Administrator/Architect
Apply on
DataStaff, Inc is seeking a Databricks Administrator/Architect for a long-term contract opportunity with one of our direct clients located in Raleigh, NC.
*This position is hybrid; may require up to 1-2 days per month for onsite meetings
Job Description:
Our client seeks a Databricks Administrator/Architect with proven skills for a 12-month engagement for creation/tuning & support of the Databricks environment. This position will be responsible for developing and designing the Databricks environment. This individual will work with internal staff to plan/design/maintain the Databricks environment and recommend changes needed to accommodate/grow as our business needs dictate. This individual will facilitate changes through the change process and work very closely with the DBA & Development Staff regarding all aspects of the design and planning of the Databricks environment.
Responsibilities:
- Provide mentorship, guidance, overall knowledge share, and support to team members, promoting continuous learning and development.
- Oversee the design, implementation, and maintenance of Databricks clusters.
- Ensure the platform s scalability, performance, and security.
- Provide escalated support and troubleshooting to users.
- Oversee maintenance of role-based access to data and features in the Databricks Platform using Unity Catalog.
- Review clusters health check and best practices implementation.
- Review and maintain documentation for users and administrators.
- Design and implement tailored data solutions to meet customer needs and use cases, spanning from ingesting data from APIs, building data pipelines, analytics, and beyond within a dynamically evolving technical stack.
- Work on projects involving on-prem data ingestion into Azure using ADF.
- Build data pipelines based on the medallion architecture that clean, transform, and aggregate data from disparate sources.
Required Skills:
- 5 Years - Extensive hands-on experience implementing Lakehouse architecture using Databricks Data Engineering platform, SQL Analytics, Delta Lake, Unity Catalog
- 5 Years - Strong understanding of Relational & Dimensional modeling.
- 6 Years - Demonstrate proficiency in coding skills - Python, SQL, and PySpark to efficiently prioritize perf, security, scalability, robust data integrations.
- 2 Years - Experience implementing serverless real-time/near real-time arch. using Cloud (i.e., Azure, AWS, or Google Cloud Platform Tech Stack), and Spark tech (Streaming & ML)
- 4 Years - Experience Azure Infra config (Networking, architect and build large data ingestion pipelines and conducting data migrations using ADF or similar tech
- 7 Years - Experience working w/ SQL Server features such as SSIS and CDC.
- 2 Years - Experience with Databricks platform, security features, Unity Catalog, and data access control mechanisms.
- 4 Years - Experience with GIT code versioning software.
Desired Skills:
- Databricks Certifications
This opportunity is available on a corp-to-corp basis or as a W2 position with a competitive benefits package. DataStaff, Inc. offers medical, dental, and vision coverage options as well as paid vacation, sick, and holiday leave. As many of our opportunities are long-term, we also have a 401k program available for employees after 6 months.