Epicareer Might not Working Properly
Learn More

Data Engineer

Salary undisclosed

Apply on


Original
Simplified

Greeting from ABAL Technologies Inc

Role: Data Engineer

Location: Pittsburgh, PA (or) Network City, NY (HYBRID)

Visa: Only H1 Transfer

NOTE: Only Candidate only, Means candidate should agree to transfer H1 on our

Note: Must have experience with ETL, Python, Spark, Hadoop

Responsibilities:

  • Organize business needs into ETL/ELT logical models and ensure dafa structures are designed for flexibility to support scalability of business solutions.
  • Craft and implement data pipelines utilizing Spark and Python
  • Define and deliver reusable components for ETL/ELT framework.
  • Define optimal data flow for system integration and data migration.
  • Integrate new data management technologies and software engineering tools into existing structures.
  • Design, build and maintain CI/CD pipelines in multiple integration and test environments.
  • Install, configure and manage automated testing tools in the environment.

Qualifications:

  • Experienced in Design, Development, and Implementation of large-scale projects in financial industries using Data Warehousing ETL tools(Spark)
  • Experience in creating ETL transformations and jobs using PySpark and automating workflows using Orchestration tools like Airflow, Control-M
  • Strong knowledge and experience of SQL, Python and Spark
  • Experience with Big Data/distributed frameworks such as Spark, Kubernetes, Hadoop, and Hive
  • Ability to design ETL/ELT solutions based on user reporting and archival requirements
  • Strong sense of customer service to consistently and effectively address client needs
  • Self-motivated; comfortable working independently under general direction
  • Hands-on experience in building and managing CI/CD pipelines

Basic knowledge of Azure Cloud Components

Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job