Epicareer Might not Working Properly
Learn More

Data Engineer

Salary undisclosed

Apply on


Original
Simplified

A **Data Engineer** is responsible for designing, building, and managing the infrastructure that enables data collection, storage, and analysis. Here's a typical job description for a **Data Engineer**:

Job Title: Data Engineer

Job Type: Full-time
Location: On-site/Remote/Hybrid

Job Summary:
The Data Engineer will design, construct, install, and maintain scalable data management systems, ensuring efficient data flow and access for business users. You will work closely with data scientists, analysts, and other stakeholders to create robust, high-performance data pipelines that enable data-driven decision-making across the organization.

Key Responsibilities:

Data Pipeline Development: Design, build, and maintain scalable data pipelines that move data from various sources (structured, unstructured) to centralized storage or databases (data lakes, warehouses).

Data Integration: Integrate data from multiple data sources and databases, transforming and loading it into a format that can be used for analytics and reporting.

Database Management: Design and implement optimized data storage solutions, such as relational databases (SQL) and NoSQL systems.

Data Quality and Governance: Ensure data accuracy, consistency, and completeness through ETL processes, data validation, and cleansing.

Optimization: Optimize performance of data retrieval and storage, ensuring data systems are scalable and performant.

Collaboration: Work closely with data scientists, business analysts, and software developers to understand data requirements and ensure smooth operation of data workflows.

Automation: Automate data collection, transformation, and processing tasks using tools like Apache Airflow, Luigi, or similar platforms.

Documentation: Create and maintain detailed documentation of data systems, workflows, and pipelines for both technical and non-technical stakeholders.

Security & Compliance: Implement and maintain data security measures to protect sensitive data, ensuring compliance with data privacy regulations (e.g., GDPR, CCPA).

Qualifications: Education: Bachelor's degree in Computer Science, Information Technology, or a related field.

Experience: 3+ years of experience in data engineering, ETL development, or a related role.

Technical Skills:

Languages: Proficiency in SQL, Python, Java, or Scala.

Database Technologies: Experience with relational (SQL, PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases.

Big Data Tools: Familiarity with Hadoop, Spark, Kafka, or similar big data technologies.

Cloud Platforms: Experience with AWS, Google Cloud Platform, Azure, or other cloud services, specifically in their data and analytics services.

ETL Tools: Experience with ETL frameworks and tools like Apache NiFi, Talend, Informatica, or Fivetran.

Data Warehousing: Familiarity with data warehouse design (e.g., Redshift, Snowflake, BigQuery).

Soft Skills:
Strong problem-solving skills and attention to detail.
Excellent communication and teamwork abilities.
Ability to work independently and prioritize multiple tasks.
Strong analytical and troubleshooting skills.

Nice-to-Have
Experience with machine learning frameworks or data science tools.
Familiarity with containerization (Docker) and orchestration (Kubernetes).
Experience with real-time data processing and stream analytics.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job