Principle Devops Engineer

We are currently looking for a Principal DevOps Engineer to join our team. This is a hands-on role in an enterprise-wide data and digital transformation initiative and deliver excellent user experience to data scientists for leveraging the Mesh system and building analytical models/insights.

The ideal candidate will be customer-centric and deeply passionate about utilizing modern cloud architectures and technology to drive substantial business impacts across organizational boundaries. The DevOps Engineer will provide support for the Analytics Workbench (AWB), specifically Snowflake & DataBricks, catering to the Business Analytics Service (BAS) team and other analytics teams that come on board. Their responsibilities will encompass the administration, maintenance, and provisioning of the infrastructure elements within Northern Trust s Azure data mesh for Business Analytics Service (BAS). A key focus of this role will involve Infrastructure as Code (IaC), where the candidate will implement technical solutions to automate the provisioning of compute cluster capacity, manage compute and storage resources, monitor resource utilization, and assist in migrating production interactive notebooks to more job-based compute capabilities.
Responsibilities:
Maintain and evolve AWB cloud-based platform capabilities (including ETL platform that will be selected to ingest data)
Collaborate with multiple internal teams (e.g. Cloud Foundation, Info Sec, Network Teams) to manage and maintain Analytics Workbench (e.g. billing, subscription levels, workspaces, Azure account hosting, audit logs, and high-level usage monitoring of platform users)
Research and prototype solutions for data connectivity and operations across various Microsoft Azure Platforms. Responsible for creating and maintaining materials covering data ingestion methods, troubleshooting, and addressing related queries.
Implement DataBricks delegated account administration across business divisions and functions, ensuring effective management and support.
Act as the key DataBricks workspace administrator, facilitating discussions, address partner needs, and providing support and planning for new implementations.
Develop and enhance the AWB platform in line with vendor release cycles and internal requirements (e.g., data protection standards). Regularly review systems and propose improvements.
Collaborate proactively with software engineers and site reliability engineers to troubleshoot Databricks performance, connectivity, and security issues within the platform engineering organization and user community.
Explore, adopt, and apply new technologies to resolve issues while adhering to company security protocols and standards.
Required Skills:
At least 12 years of experience in developing and deploying large scale data solution in an enterprise and/or cloud environment.
A minimum of 2 years hands-on staging and production experience in each of the following areas
Microsoft Azure DataBricks
ADLS
Data Ingestion and ETL (e.g. ADF/Azure Synapse)
Snowflake
Documentation and Troubleshooting- Documentation and Troubleshooting
In-depth operational expertise in the following areas:
Cloud architecture best practices around operational excellence, security, reliability, performance efficiency, and cost optimization (e.g. Cloud Well Architected Framework)
Best practices and IT operations for dynamic, always-up, always-available services
Bachelor's degree in Computer Science or related discipline.
Preferred Skills:
Knowledge of data analytics technology and methodology, such as advanced analytics or machine learning
Experience with DevOps, DataOps, and/or MLOps using ADO
Working knowledge of data security such as SecuPi, and Azure native capabilities
Knowledge of IaC and automation such as Terraform and Powershell
Working experience with Azure and AWS cloud data and service offerings

Additional Skills & Qualifications

Knowledge of data analytics technology and methodology, such as advanced analytics or machine learning
Experience with DevOps, DataOps, and/or MLOps using ADO
Working knowledge of data security such as SecuPi, and Azure native capabilities
Knowledge of IaC and automation such as Terraform and Powershell
Working experience with Azure and AWS cloud data and service offerings

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.