Sr. Cloud Data Engineer

Only local San Francisco Bay Area candidates only!

Direct W2 candidates only! No 3rd party agencies!

Job Posting Title

Sr Cloud Data Engineer

Description

Job Title: Sr Cloud Data Engineer
Location: South San Francisco, CA - Hybrid
Job Type: Contract, Full-Time Job Description:
We are seeking a highly skilled and motivated Cloud Data Engineer to join our growing data team. The ideal candidate will have a strong background in developing data pipelines, implementing data models, and executing best practices for developing data products. This role requires expertise in AWS, GitLab CI/CD, dbt, Snowflake, SQL, Python, Git, and DevOps, as well as hands-on experience with orchestrators such as Airflow and AutoMateNow, and data governance technologies like Monte Carlo and Collibra. Key Responsibilities:
Develop Data Pipelines: Design, develop, and maintain robust, scalable, and efficient ETL/ELT pipelines to support diverse data sources and large-scale data processing.
Data Modeling: Create and maintain data models and data architecture to ensure data integrity and optimum performance.
Best Practices Implementation: Apply industry best practices in data warehousing, data governance, and data lifecycle management.
Collaboration: Work closely with data scientists, analysts, and other stakeholders to gather requirements and deliver high-quality data products.
Automation and CI/CD: Implement CI/CD pipelines using GitLab for automated testing, deployment, and integration of data workflows.
Database Management: Manage and optimize Snowflake databases for performance, scalability, and cost-effectiveness.
Coding and Scripting: Write efficient SQL queries and Python scripts to extract, load, and transform data.
Version Control: Utilize Git for version control, ensuring traceable and manageable code changes.
Orchestration: Utilize orchestration tools like Airflow and AutoMateNow to manage and automate data workflows.
Data Governance: Implement and manage data governance and data observability solutions using technologies like Monte Carlo and Collibra.
Monitoring and Optimization: Monitor data pipeline performance and troubleshoot issues to ensure reliability and efficiency.
Documentation: Maintain comprehensive documentation for data pipelines, models, and processes. Qualifications:
Education: Bachelor s degree in Computer Science, Information Technology, Data Science, or a related field. Experience:
Minimum 5 years of experience in data engineering or a similar role.
Proven experience with AWS cloud services related to data processing, such as S3, Redshift, Lambda, Glue, and Data Pipeline. Technical Skills:
Proficiency in designing and implementing CI/CD pipelines with GitLab.
Experience with dbt (data build tool) for transforming data within the warehouse.
Strong knowledge of Snowflake with hands-on experience in data warehousing solutions.
Expertise in SQL for data querying, manipulation, and optimization.
Proficiency in Python for scripting and automation tasks.
Familiarity with DevOps practices and tools.
Experience with orchestrators like Airflow and AutoMateNow.
Knowledge of data governance and observability tools such as Monte Carlo and Collibra.
Strong working knowledge of Git for version control. Soft Skills:
Strong problem-solving and analytical skills.
Excellent communication skills, both written and verbal.
Ability to work collaboratively in a fast-paced, dynamic environment.
High attention to detail and commitment to producing high-quality work. Preferred Qualifications:
AWS Certified Data Analytics Specialty or AWS Certified Solutions Architect.
Experience with real-time data processing and streaming technologies (e.g., Kafka, Kinesis).
Experience in a regulated industry
Experience in a manufacturing environment

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.