Sr. Lead Data Engineer _ Need Locals at Scottsdale AZ (onsite 3Days a week) _ 10+ Years
Apply on
Requirement Details:
Role:- Sr. Lead Data Engineer
Location : Scottsdale AZ(3Days a week)
Duration: 12+ Months
HAS TO SIT ONSITE 3 times a week in Scottsdale AZ
Healthcare experience is Mandatory
Job Description:
Looking for a Lead Data Engineer to assist us in solutioning architecture as we transform the Rebate Processing System from its legacy platform. This person will assist our software engineers in rearchitecting, transforming, and designing data pipelines.
Desired skills: Map Reduce, Spark Streaming, PySpark, Hive, HDFS, Kafka, Redis, Sqoop, Oozie, IBM Data Sage, Spring Batch
Required skills:
10+ Years of IT Experience
Healthcare experience is required
5-7 years of data engineering experience
Capable of leading other data engineers
3+ years of experience with Apache Airflow and/or Composer
3+ years of experience with DataProc and/or Spark
5+ years of experience with CI/CD
Experience producing architecture solutions using Composer and Spark
Experience in large-scale batch processing with cloud services
Ability to collaborate with architects to solve problems and provide data engineering solutions
Familiarity with databases and SQL
Familiarity with at least one OOP language, Java preferred
Familiarity with Google Cloud Platform, GKE, and Cloud IAM