Data Engineer / Hadoop Developer

Job Title: Data Engineer

Location: Jersey City, NJ

Duration: Full Time

Job Responsibilities:

Spark & Scala Expertise:

Strong foundation in Scala: The primary language for Spark development, so a solid understanding of its syntax, data structures, functional programming, and common libraries is critical.

Extensive Spark experience: Deep knowledge of Spark core concepts (RDDs, DataFrames, Datasets), Spark SQL, and Spark Streaming for building robust and performant data pipelines.

Familiarity with Spark cluster deployment: Understanding how to configure and run Spark applications in a cluster environment (e.g., YARN, Mesos).

ETL Development:

Data Extraction: Experience with various data sources (relational databases, APIs, flat files, etc.) and techniques for extracting data from them.

Data Transformation: Mastery of data manipulation techniques using Spark (aggregation, filtering, joining, etc.) to prepare data for analysis and loading.

Data Loading: Experience with loading data into various target systems, including Hive tables, databases, and data lakes.

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.