Epicareer Might not Working Properly
Learn More

Azure Data Engineer

Salary undisclosed

Apply on


Original
Simplified

Our goal is to solve problems and deliver results for our clients. At Cloud and Things, you can be a part of transforming the public sector's IT environment. Our team is on the forefront of helping to solve the government's most complex IT challenges. If you are seeking a role that offers the opportunity to work on rewarding projects, consider a career with Cloud and Things.

Overview:

We are seeking a Data Engineer to join our team for a healthcare project supporting one of our clients. We are looking for a skilled Azure Data Engineer who is detail oriented. In this role you will be responsible for building and managing data engineering pipelines.

Duties:

  • Design, build, and manage Azure Data Factory pipelines for data ingestion.

  • Handle both batch and near real-time data ingestion using Azure Service Bus.

  • Integrate structured, semi-structured, and unstructured data sources.

  • Utilize formats such as Parquet, Avro, JSON, etc., and manage loading into the Bronze layer.

  • Build and manage data engineering pipelines using Databricks (PySpark, SparkSQL).

  • Implement Spark-based data transformation jobs from Bronze to Silver and Gold layers.

  • Establish and manage Data Quality Framework using Databricks and Data Factory.

  • Set up data security using RBAC and ACLs to manage permissions on sensitive datasets.

  • Implement automation frameworks for pipeline execution using Databricks Workflows.

  • Monitor pipelines using Azure Monitor, set up alerts and ensure SLAs are met.

  • Optimize storage in ADLS Gen2 and use Azure Key Vault to secure credentials.

  • Optimize costs using Azure Cost Management.

Mandatory Qualifications:

  • Bachelor's degree

  • Azure Data Factory for ETL/ELT operations.

  • Databricks (PySpark, SparkSQL) for data transformations.

  • Azure Data Lake Storage (ADLS Gen2).

  • SQL Server, PostgreSQL, Cosmos DB, CRM, and ERP systems.

  • Data Governance, RBAC, and ACLs for managing permissions.

  • CI/CD tools: Azure DevOps, GitHub Actions, Terraform.

  • Azure Monitoring, Alerting, and Logging tools.

Desirable Qualifications:

  • Master's degree preferred

Cloud and Things complies with all applicable federal, state, and local laws regarding recruitment and hiring. All qualified applicants are considered for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, protected veteran status, or any other category protected by applicable federal, state, or local laws.

Cloud and Things complies with all applicable federal, state, and local laws regarding recruitment and hiring. All qualified applicants are considered for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, disability, protected veteran status, or any other category protected by applicable federal, state, or local laws.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job