G
Senior Data Engineer (Hybrid)
Salary undisclosed
Apply on
Original
Simplified
Job Description
Job Description
NOTE: The client expects, that the Candidate will be visiting the office 4 days per week (Toronto,ON)
NOTE: The client expects, that the Candidate will be visiting the office 4 days per week (Toronto,ON)
Hudi - Experience is a must
Responsibilities
Manage timelines/deliverables within the team towards the successful delivery of projects.
Design software solutions by interacting with portfolio managers, traders, operations staff and peers to understand requirements.
Develop solutions that are in line with client's technology biases, deliver efficiency and scalability, and enable new trading activities.
Provide knowledge transfer to team members and support staff through application demos, walkthroughs, and documentation.
Design software solutions by interacting with portfolio managers, traders, operations staff and peers to understand requirements.
Develop solutions that are in line with client's technology biases, deliver efficiency and scalability, and enable new trading activities.
Provide knowledge transfer to team members and support staff through application demos, walkthroughs, and documentation.
Skills Must have
Concepts: Data Mesh, Data Product
Basic :Data Pipelines: PySpark, Hudi, Iceberg, Airflow, EMR
Expert :Data Operations: Operational Excellence, monitoring, troubleshooting
Expert :Data Quality: Great Expectations, AWS Glue DQ, Deequ
Expert :Entitlements: AWS Lakeformation
Expert :Infrastructure: AWS Data Eng. Services (Glue, EMR, Athena etc.), AWS Compute Services (EC2, Lambda), AWS Storage Service (S3), cross-account IAM, RAM, MWAA, Step Functions, etc.
Knowledge :DevOps: Terraform, Git Actions
Expert :Metadata: Neptune, OpenLineage, Angular, DataZone
Consumption: AWS Athena, Trino
Other Tools: JupyterLab, Databricks
Concepts: Data Mesh, Data Product
Basic :Data Pipelines: PySpark, Hudi, Iceberg, Airflow, EMR
Expert :Data Operations: Operational Excellence, monitoring, troubleshooting
Expert :Data Quality: Great Expectations, AWS Glue DQ, Deequ
Expert :Entitlements: AWS Lakeformation
Expert :Infrastructure: AWS Data Eng. Services (Glue, EMR, Athena etc.), AWS Compute Services (EC2, Lambda), AWS Storage Service (S3), cross-account IAM, RAM, MWAA, Step Functions, etc.
Knowledge :DevOps: Terraform, Git Actions
Expert :Metadata: Neptune, OpenLineage, Angular, DataZone
Consumption: AWS Athena, Trino
Other Tools: JupyterLab, Databricks
Good to have
SQL We are looking for experienced support whereby experience means comfortable working with production-level code and assets with clear understanding of DevOps and DataOps
SQL We are looking for experienced support whereby experience means comfortable working with production-level code and assets with clear understanding of DevOps and DataOps
Employers have access to artificial intelligence language tools (“AI”) that help generate and enhance job descriptions and AI may have been used to create this description. The position description has been reviewed for accuracy and Dice believes it to correctly reflect the job opportunity.
Report this job Similar Jobs