AWS Data Engineer - 100% Remote - 12+ Months Contract - Direct Client

Job Title: AWS Data Engineer

Duration: 12+ Months Contract

Location: 100% Remote

Job description :

Responsibilities

• Collaborate closely with cross-functional teams including Data Scientists, Analysts, and Software Engineers to understand data requirements and translate them into efficient solutions using Snowflake, Databricks, Kafka, AWS Glue, Airflow, and AWS services.
• Design and implement end-to-end data pipelines, leveraging Kafka for real-time data streaming, Snowflake for scalable data warehousing, and Databricks for advanced analytics.
• Develop and maintain ETL processes using AWS Glue to facilitate seamless and reliable data extraction, transformation, and loading into Snowflake and Databricks.
• Optimize data models and schema designs for improved query performance within Snowflake and Databricks, while ensuring data consistency and integrity.
• Implement robust data security measures and access controls in alignment with company policies and industry best practices.
• Utilize Airflow to orchestrate and schedule data workflows, ensuring timely and accurate data processing.
• Monitor, troubleshoot, and enhance data pipelines, identifying and resolving performance bottlenecks, data quality issues, and other challenges.
• Contribute to the evolution of data warehousing strategies, encompassing data partitioning, clustering, and distribution within Snowflake and Databricks.
• Stay up-to-date with the latest advancements in Snowflake, Databricks, Kafka, AWS Glue, Airflow, and AWS services, and advocate for their effective utilization within the organization

Experience/Minimum Requirements

• Proven experience 8+ years as a Data Engineer, with a strong emphasis on Snowflake, Databricks, Kafka, AWS Glue, Airflow, and AWS services.
• In-depth understanding of Snowflake architecture, performance optimization techniques, and best practices.
• Proficiency in SQL and experience with database design principles.
• Hands-on expertise in designing, building, and maintaining complex ETL pipelines using Snowflake, Databricks, Kafka, and AWS Glue.
• Familiarity with data warehousing concepts and methodologies.
• Strong grasp of real-time data streaming principles and experience with Kafka.
• Competence in cloud computing and AWS services, with a focus on data-related services such as S3, Redshift, and Lambda.
• Proficiency in scripting and programming languages such as Python, Java, or similar.
• Relevant certifications in Snowflake, Databricks, AWS, or related technologies are advantageous

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.