Epicareer Might not Working Properly
Learn More
Please Turn on your JavaScript

Epicareer might not working properly because your browser javascript is turned off.

What is JavaScript?

JavaScript is like the magic behind making websites interactive. Basically, it makes Epicareer more enjoyable and engaging.
How do i turn this on?
Find tutorial for your browser below :
  1. Make sure the webpage is open.
  2. In the Safari app on your Mac, choose Safari > Preferences, then click Security.
  3. Make sure the Enable JavaScript checkbox is selected.
  4. Click Websites.
  5. On the left, click Content Blockers.
  6. Make sure Off is chosen in the pop-up menu next to the website.
  7. On the left, click Pop-up Windows.
  8. Make sure Allow is chosen in the pop-up menu next to the website.
Visit Site here
  1. Open Chrome on your computer.
  2. Click. then Settings.
  3. Click Privacy and Security.
  4. Click Site settings.
  5. Click JavaScript.
  6. Select Sites can use Javascript.
Visit Site here
  1. If you're running Windows OS, in the Firefox window, click Tools > Options.
  2. Tip: If you're running Mac OS, click the Firefox drop-down list > Preferences.
  3. On the Content tab, click the Enable JavaScript check box.
Visit Site here
  1. Click the "Settings and more" icon at the top right of the Edge browser window.
  2. Click Settings.
  3. From the Settings menu, click Cookies and site permissions.
  4. Scroll down and click on JavaScript.
  5. In the JavaScript window, toggle the slider to "on" or "allowed."
  6. Close the Settings tab.
  7. Refresh the page.
Visit Site here
  1. Go to Settings.
  2. Click Advanced in the left sidebar, and click Privacy & security.
  3. Under Privacy and security, click Site settings.
  4. Click JavaScript.
  5. At the top, turn on or off Allowed (recommended).
Visit Site here
Any Question? Contact us
logo
Jobs or company...
E

Spark Developer / Engineer

Salary undisclosed
Checking job availability...
Original
Simplified

Job Title: Spark Developer / Engineer (2 positions)

Location: US Remote, work during PST time zone

Duration: 6-12 Months

Workflows are powered by offline batch jobs written in Scalding, a MapReduce-based framework. To enhance scalability and performance, migrating these jobs from Scalding to Apache Spark.

Key Responsibilities:

Understanding the Existing Scalding Codebase

o Analyze the current Scalding-based data pipelines.

o Document existing business logic and transformations.

Migrating the Logic to Spark

o Convert existing Scalding jobs into Spark (PySpark/Scala) while ensuring optimized performance.

o Refactor data transformations and aggregations in Spark.

o Optimize Spark jobs for efficiency and scalability.

Ensuring Data Parity & Validation

o Develop data parity tests to compare outputs between Scalding and Spark implementations.

o Identify and resolve any discrepancies between the two versions.

o Work with stakeholders to validate correctness.

Writing Unit Tests & Improving Code Quality

o Implement robust unit and integration tests for Spark jobs.

o Ensure code meets engineering best practices (modular, reusable, and well-documented).

Required Qualifications:

  • Experience in big data processing with Apache Spark (PySpark or Scala).
  • Strong experience with data migration from legacy systems to Spark.
  • Proficiency in Scalding and MapReduce frameworks.
  • Experience with Hadoop, Hive, and distributed data processing.
  • Hands-on experience in writing unit tests for Spark pipelines.
  • Strong SQL and data validation experience.
  • Proficiency in Python, Scala
  • Knowledge of CI/CD pipelines for data jobs.
  • Familiarity with Apache Airflow orchestration tool.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job

Job Title: Spark Developer / Engineer (2 positions)

Location: US Remote, work during PST time zone

Duration: 6-12 Months

Workflows are powered by offline batch jobs written in Scalding, a MapReduce-based framework. To enhance scalability and performance, migrating these jobs from Scalding to Apache Spark.

Key Responsibilities:

Understanding the Existing Scalding Codebase

o Analyze the current Scalding-based data pipelines.

o Document existing business logic and transformations.

Migrating the Logic to Spark

o Convert existing Scalding jobs into Spark (PySpark/Scala) while ensuring optimized performance.

o Refactor data transformations and aggregations in Spark.

o Optimize Spark jobs for efficiency and scalability.

Ensuring Data Parity & Validation

o Develop data parity tests to compare outputs between Scalding and Spark implementations.

o Identify and resolve any discrepancies between the two versions.

o Work with stakeholders to validate correctness.

Writing Unit Tests & Improving Code Quality

o Implement robust unit and integration tests for Spark jobs.

o Ensure code meets engineering best practices (modular, reusable, and well-documented).

Required Qualifications:

  • Experience in big data processing with Apache Spark (PySpark or Scala).
  • Strong experience with data migration from legacy systems to Spark.
  • Proficiency in Scalding and MapReduce frameworks.
  • Experience with Hadoop, Hive, and distributed data processing.
  • Hands-on experience in writing unit tests for Spark pipelines.
  • Strong SQL and data validation experience.
  • Proficiency in Python, Scala
  • Knowledge of CI/CD pipelines for data jobs.
  • Familiarity with Apache Airflow orchestration tool.
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job