Data Engineer 2 - Applicants must be US Citizens with TS - FSP to apply
Apply on
Job Description
Requirements:
Design and optimize Data Pipelines using Spark, Hudi, EMR cloud services, and Kubernetes containers
Make sure pedigree and provenance of the data is maintained such that the access to data is protected
Clean and preprocess data to enable analytic access
Collaborate with enterprise working groups to advance the state of data standards
Collaborate with the engineering team, data stewards, and mission partners to aid in
getting actionable value out of the data holdings architects complex, repeatable ETL
processes
Provide Advanced Database Administration support in Oracle, MySQL, MariaDB, MongoDB,
Elastic and others
Supports Experience with Targeting using Sponsor Tools, Reverse Engineering
Develop API connectors to enable ingest of new data catalog entries from databases and files
Ensure that data mappings will provide the best performance for expected user experience